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PREFACE 


The  papers  contained  in  this  volume  were  prepared  for  discussion  at  the  Third 
Congress  on  Information  System  Science  and  Technology.  The  Congresses, 
established  as  biennial  events,  were  formed  to  increase  communication  among 
scientists,  engineers,  and  military  personnel  in  the  science  and  technology  of 
information  systems.  The  Third  Congress,  scheduled  for  the  21st  and  22nd  of 
November,  1966,  was  postponed  in  response  to  a  presidential  directive  to  curtail 
such  expenditures.  However,  the  timeliness  of  the  papers  makes  it  desirable  to 
issue  them  in  their  present  form.  Introductory  remarks  by  the  session  chairmen 
have  been  included  where  possible  to  show  the  context  in  which  the  papers  were 
to  have  been  discussed. 

It  has  been  customary  to  issue  the  papers  for  the  Congresses  in  a  preprint  form 
and  to  devote  the  sessions  exclusively  to  discussions  of  their  content.  In  con¬ 
trast,  three  of  the  sessions  at  the  Third  Congress  were  planned  with  the  expecta¬ 
tion  that  papers  would  emerge  as  products  of  the  discussions  rather  than  as 
stimuli  for  them.  Two  of  these  sessions  are  represented  by  papers,  but  the  third 
session  is  identified  only  by  summary. 
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Some  brainware  problems  in  information  systems 
and  operations  analysis 


by  Lewis  C.  Clapp 

Computer  Research  Corporation 

Newton,  Massachusetts 

INTRODUCTION 

The  basic  theme  of  this  paper  is  quite  simple;  namely, 
we  do  not  always  have  sufficient  experience  to  extrap¬ 
olate  the  characteristics  of  small-scale  information 
systems  to  produce  the  effective  large-scale  systems 
which  are  in  demand  today.  Our  problem  does  not 
rest  only  with  the  hardware  and  software  techniques 
which  are  needed  to  build  these  systems,  but  also  with 
the  brainware  considerations  which  must  come  long 
before  the  hardware  and  the  software  can  be  intelli¬ 
gently  integrated  to  produce  the  required  system. 

Lack  of  communication  between  the  client,  that  is, 
the  man  who  will  use  the  system,  and  the  system  de¬ 
signer  is  the  first  aspect  of  the  brainware  problem. 
Often  the  client  does  not  know  how  to  state  his  infor¬ 
mation  problem  in  terms  that  the  system  designer  can 
use  and  vice  versa.  In  fact  the  client  and  the  system 
designer  may  be  using  identical  words  and  talking 
about  distinctly  different  things.  While  everyone  as¬ 
sociated  with  the  development  of  an  information  sys¬ 
tem  must  share  the  burden  and  responsibility  for  com¬ 
munication  during  the  initial  systems  analysis,  the  in¬ 
formation  system  designer  should  be  the  person  most 
capable  of  handling  the  problem.  But  as  practitioners 
of  information  science,  we  often  fail  to  communicate 
accurately  even  among  ourselves. 

We  would  not  put  a  blindfolded  man  into  a  car  and 
force  him  to  drive  down  the  Los  Angeles  Freeway  at 
80  miles  an  hour.  Yet,  we  may  be  performing  an 
analagous  act  as  we  develop  large-scale  management 
information  systems  for  the  man  who  is  ill-prepared 
to  use  such  tools  wisely  and  with  appropriate  human 
judgment. 

Consider,  for  example,  the  manager  who  enters 
his  office  in  the  morning,  reads  reports,  examines 
computer  displays,  and  then  makes  major  decisions 
based  on  information  processed  by  a  large-scale  com¬ 
puter  information  system.  If  this  manager  is  typical, 
he  will  have  but  a  vague  idea  of  how  the  information 
was  gathered,  fed  into  the  computer,  and  then  pro¬ 


cessed  to  produce  the  reports  upon  which  he  bases 
his  judgment.  What  goes  on  in  the  computer  is  a 
black  mystery  to  this  manager.  Our  typical  manager 
is  in  the  dark  first,  because  he  doesn’t  understand 
what  the  system  really  does  and  therefore  he  cannot 
use  the  results  selectively  and  second,  because  the 
system  may  have  inherent  faults  for  which  he  cannot 
make  proper  allowances.  Poor  communication  of 
problems,  objectives  and  techniques  lies  at  the  root 
of  the  difficulty. 

It  is  both  natural  and  necessary  for  each  field  to 
develop  its  own  peculiar  technical  terms  and  jargon 
to  facilitate  communication  among  workers  in  that 
field.  But  the  jargon  of  our  field  does  not  excuse  us 
from  our  responsibility  to  communicate  with  our 
customers.  Indeed,  it  is  rather  remarkable  to  see 
words  and  concepts  used  in  an  imprecise  and  some¬ 
times  misleading  manner  in  an  industry  whose  main 
concern  is  the  processing  of  information  and  the  com¬ 
munication  of  data.  Consider,  for  example,  the  four 
key  words  in  the  title  of  this  session:  Information 
Systems  and  Operations  Analysis. 

Each  word  individually  is  so  broad  and  vague  that 
it  transmits  very  little  information  about  the  subject 
at  hand.  In  fact  “information”  and  “system”  are  two 
of  the  most  overworked  words  in  the  English  language, 
preceded  in  frequency  of  usage  only  by  the  pronoun 
“I.”  Even  after  the  words  are  combined  in  pairs  we 
are  not  much  better  off.  What  is  an  information  sys¬ 
tem,  or  operations  analysis  for  that  matter?  And  con¬ 
sidering  the  title  as  a  whole:  Are  we  talking  about 

(1)  operations  analysis  on  information  systems, 

(2)  information  systems  for  operation  analysis,  or 

(3)  information  systems  and  operation  analysis  ap¬ 
plied  to  some  other,  as  yet  undefined,  thing?  We  use 
words  such  as  these  as  if  there  were  universal  con¬ 
sensus  on  their  meaning,  while  privately  each  of  us 
has  conjured  up  a  different  definition.  Meanwhile 
that  manager,  who  will  be  the  ultimate  user  of  the 
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“system,”  looks  on  impressed  by  our  vocabulary  but 
confused  about  what  we  are  doing  for  him. 

Perhaps,  two  sets  of  vocabulary  are  needed,  one 
for  internal  communication  within  the  field,  and  a 
second  to  facilitate  clear  and  simple  communication 
with  the  man  who  pays  the  bills  and  has  to  live  with 
the  product  of  our  labors.  It  might  even  turn  out  that 
the  layman’s  vocabulary  could  be  used  profitably 
among  workers  within  the  information  science  field. 

What  do  we  mean  by  information? 

Several  years  ago  a  young  M.I.T,  student  took  his 
girl  friend  (who  was  not  a  science  major)  to  hear  a 
lecture  on  vacuum  technology  by  the  great  physicist 
Enrico  Fermi.  As  they  were  strolling  home  the  young 
physicist  asked  his  friend  if  she  had  learned  anything 
from  the  lecture  which  had  clearly  been  over  her  head. 
“Yes,”  she  replied,  “I  learned  two  things.  One,  Pro¬ 
fessor  Fermi  cannot  draw  a  straight  line  and  two,  the 
vacuum  is  where  the  air  is  not.”  As  with  the  vacuum, 
it  is  easier  to  explain  what  information  is  not,  rather 
than  what  it  is. 

Our  difficulty  in  producing  an  operational  defini¬ 
tion  of  the  word  “information,”  may  be  due  in  part 
to  the  fact  that  the  concept  is  dynamic  and  changing 
constantly.  Only  a  short  time  ago  information  to  a 
computer  man  meant  numeric  or  printed  data  (e.g., 
words  of  text).  Gradually  our  ability  to  handle  other 
types  of  information  increased.  Today,  for  example, 
we  are  making  good  progress  in  processing  many 
types  of  information  including  graphical  data  such  as 
line  drawings,  photographs,  and  even  analog  signals 
(e.g.,  acoustic  signals).  With  these  new  techniques 
for  handling  specific  problems  came  the  need  for 
novel  types  of  input/output  devices  to  handle  these 
other  types  of  information.  The  result  has  been  im¬ 
portant  gadgets  such  as  display  scopes  and  light- 
pens,  Rand  Tablets  and  film  readers.  Our  progress 
in  the  area  of  applications  has  been  impressive,  but 
what  of  our  achievements  in  the  area  of  fundamental 
theory? 

There  have  been  several  attempts  made  at  develop¬ 
ing  a  useful  mathematical  description  of  the  informa¬ 
tion  handling  process  but  these  have  not,  in  general, 
met  with  much  practical  success.  For  example,  the 
operational  definition  of  information  in  Shannon’s 
Information  Theory  is  too  restrictive  to  be  useful 
in  the  information  processing  field  (except  for  prob¬ 
lems  associated  with  information  transfer  and  com¬ 
munication).  To  the  best  of  our  knowledge,  no  one 
has  yet  developed  a  completely  satisfactory  theory 
of  information  processing.  Because  there  is  no  strong 
theoretical  basis  for  the  field,  we  must  rely  on  intui¬ 
tion,  experience  and  the  application  of  heuristic  no¬ 


tions  each  time  we  attempt  to  solve  a  new'  information 
processing  problem.  Frequently,  we  may  conceive 
of  several  alternate  solutions  to  the  problem,  but  we 
do  not  have  the  techniques  to  determine  which  of 
these  solutions  is  optimum  in  the  general  case. 

Although  we  have  labeled  this  field  which  has  to  do 
with  the  processing  of  information  as  “information 
science,”  wc  should  recognize  that  it  is  not  a  science 
in  the  strict  sense  of  the  word  as  are  mathematics 
and  physics.  There  are  no  axioms  or  premises  from 
which  all  conclusions  must  unavoidably  and  logically 
follow.  There  aren’t  even  simple  criteria  to  determine 
the  success  or  validity  of  a  new  technique  in  a  real 
information  system.  Each  new  system  is  a  unique  and 
special  case.  In  planning  the  system,  the  skillful  de¬ 
signer  may  select  from  a  number  of  techniques  which 
were  successful  in  earlier  systems.  But  unless  the  new 
application  is  quite  similar  to  the  old,  the  designer 
cannot  be  certain  that  the  new  system  will  function  as 
originally  conceived.  He  will  generally  rely  on  his 
ability  to  make  small  changes  in  the  design  as  the 
system  is  constructed  and  as  he  observes  the  behavior 
of  the  working  system.  Usually  these  design  changes 
(sometimes  called  field  improvements)  can  be  handled 
at  relatively  low  expense  at  the  software  level,  but 
sometimes  the  required  changes  must  be  performed 
on  the  hardware  at  great  cost.  Thus,  “information 
science”  is  more  of  an  art  than  a  science. 

Information  system  design  would  be  less  artful  and 
more  scientific  if  the  process  of  extrapolating  tech¬ 
niques  from  the  small  to  the  large  system  could  be 
carried  out  with  a  high  assurance  of  success.  But  we 
frequently  run  into  the  problem  that  techniques  which 
work  well  in  smaller  systems  where  the  data  files  are 
quite  manageable,  will  not  work  effectively  when  ex¬ 
tended  to  bigger  systems  which  must  cope  with  larger 
quantities  of  data.  For  example,  many  information  re¬ 
trieval  techniques,  which  seem  very  promising  when 
tested  with  small  files  containing  only  a  few  hundred 
items,  break  down  when  tested  with  larger  files  con¬ 
taining  tens  of  thousands  of  items.  In  the  information 
systems  business,  we  lack  one  of  the  pillars  of  the 
scientific  method;  the  reproducibility  of  results.  The 
physicist  knows  that  water  will  boil  at  2 1 2°F,  whether 
the  quantity  of  water  is  one  ounce,  one  quart  or  500 
gallons,  but  the  information  analyst  has  no  assurance 
that  a  technique  which  works  for  50  items  will  still 
work  with  a  file  of  5,000  entries.  In  information  pro¬ 
cessing  we  have  to  be  content  with  probabilities  rather 
than  certainties. 

Therefore,  the  process  of  constructing  an  informa¬ 
tion  system  generally  boils  down  to  a  cut  and  dry  pro¬ 
cedure.  The  system  is  first  designed,  implemented  and 
then  tested.  During  the  testing  phase  errors  and  in- 
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consistencies  are  found  and  then  eliminated.  Once 
again  the  system  is  tested  and  the  process  is  repeated 
until  either  the  system  works  to  everyone’s  satisfac¬ 
tion  or  the  designers  arc  thoroughly  exhausted.  Conse¬ 
quently,  information  systems  evolve  as  successive 
improvements  to  earlier  less  refined  systems.  Al¬ 
though  excellent  systems  may  result  from  the  process, 
we  must  recognize  that  this  type  of  design  process  is 
time  consuming,  inefficient,  and  costly. 

The  solution  to  this  dilemma  will  not  come  about 
by  any  great  innovations  in  hardware  or  software. 
These  are  only  gimmicks  which  reflect  the  state  of 
our  brainware.  If  our  concepts  are  sound,  coherent 
and  well  thought  out,  the  software  and  hardware 
which  mirrors  these  concepts  will  be  useful.  But  if 
our  brainware  is  haphazard  and  confused,  this  will  be 
reflected  in  the  information  systems  which  result. 

The  state  of  operations  analysis 

In  many  respects  operations  analysis  is  in  the  same 
predicament  as  information  systems  design.  Here 
we  shall  assume  that  the  objective  of  operations 
analysis  is  ( 1 )  to  gather  information  about  some  opera¬ 
tion  system  generally  boils  down  to  a  cut  and  dry  pro¬ 
potential  problems,  and  (3)  to  analyze  the  probable 
results  of  proposed  solutions  to  these  problems.  The 
operations  analyst  has  a  number  of  tools  at  his  dis¬ 
posal  among  which  include,  techniques  of  operation 
research,  mathematical  programming,  critical  path 
analysis  and  simulation.  There  is  no  general  theory 
for  solving  operations  analysis  problems  but  rather 
there  exists  a  collection  of  techniques  which  have 
worked  well  for  special  cases  in  the  past.  Here  again 
there  arc  no  general  principles  or  axioms  which  the 
operations  analyst  can  apply  to  any  new  problem 
area.  He  must,  instead,  skillfully  adapt  techniques 
which  have  apparently  worked  for  earlier  problems. 

Abstruse  mathematics  is  not  in  itself  the  answer 
to  our  difficulties.  The  complex  systems  generally 
encountered  in  the  real  world  do  not  lend  themselves 
to  neat  mathematical  formulations  and  in  most  cases 
the  operations  analyst  is  forced  to  reduce  the  problem 
to  simpler  terms  to  make  it  tractable.  This  is  often 
done  by  building  a  simplified  model  of  the  real  world 
process  under  study.  Frequently  this  model  is  simu¬ 
lated  on  a  computer  to  handle  the  high  data  volume 
in  a  reasonable  period  of  time.  It  is  important  to  verify 
that  the  solution  obtained  by  use  of  the  model  corre¬ 
sponds  to  an  acceptable  solution  in  the  real  world 
which  is  considerably  more  complicated  than  the 
model. 

In  this  paper  we  arc  concerned  with  union  of  opera¬ 
tions  analysis  and  information  systems.  The  data 
about  the  process  under  observation  is  gathered. 


processed  and  displayed  by  means  of  the  information 
system.  Just  as  the  operations  analyst  had  to  make 
some  approximations  about  the  real  world  process, 
the  data  in  the  information  system  itself  may  be  an 
approximation  to  the  real  world  because  of  the  lack 
of  complete  information.  First,  all  of  the  information 
about  the  real  world  process  may  not  be  available 
on  a  timely  basis.  Second,  the  hardware  portion  of 
the  system  has  inherent  limitations,  it  usually  cannot 
store  all  of  the  information  that  is  available  or  process 
it  in  a  reasonable  period  of  time.  Third,  we  may  not 
have  developed  suitable  software  techniques  for  pro¬ 
cessing  all  of  the  information  even  if  it  were  available 
and  could  be  handled  by  the  hardware.  Thus,  we  are 
usually  in  the  position  of  having  only  incomplete  in¬ 
formation  about  the  real  world  process  at  our  com¬ 
mand. 

Given  these  two  imperfect  or  approximate  models 
of  the  real  world  process,  one  the  result  of  the  limita¬ 
tions  of  information  science,  the  other  due  to  the  in¬ 
herent  approximations  of  operations  analysis,  there 
is  a  possibility  that  the  combined  errors  will  lead  to 
serious  deficiencies  in  the  total  system.  It  is  as  if  we 
were  simultaneously  wearing  two  pairs  of  eyeglasses 
and  trying  to  look  at  the  world.  Unless  the  two  pairs 
of  glasses  are  carefully  matched  to  one  another,  our 
view  must  be  distorted.  And  if  wc  rely  too  heavily 
on  the  total  system  in  our  planning  and  decision  mak¬ 
ing,  we  deserve  the  fate  which  must  inevitably  follow. 

Can  we  improve  our  brainware? 

The  problem  with  our  brainware  involves  funda¬ 
mental  concepts  and  principles  and  there  can  be  no 
simple  road  that  we  can  follow  to  reach  the  desired 
destination.  Newton  did  not  arrive  at  his  three  laws 
of  motion  by  saying  “I  think  I  shall  try  to  understand 
the  fundamental  nature  of  mechanics  in  the  universe, 
today.”  But  a  study  of  the  way  that  fundamental 
notions  came  about  in  many  of  the  sciences  can  at 
least  help  to  guide  us. 

First,  we  must  realize  that  it  is  the  basic  notions 
that  we  are  after.  We  must  not  allow  ourselves  to  be 
distracted  by  cute  devices  or  clever  computer  pro¬ 
grams.  The  programs  and  the  devices  are  important 
and  worthwhile,  but  they  will  usually  not  help  us  with 
tomorrow's  problems.  We  should  be  willing  to  invest 
the  time  and  money  in  a  pursuit  of  basic  understand¬ 
ing  that  may  have  little  payoff  in  the  next  few  years 
and  which  may  be  absolutely  essential  five  years 
from  now. 

Second,  wc  must  be  patient.  Given  a  new  problem 
we  should  learn  to  control  our  urge  to  jump  to  the 
solution.  Implementing  the  first  system  that  comes 
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into  our  heads  is  definitely  the  most  costly  way  to  go 
about  solving  an  important  problem.  We  should  first 
consider  the  simplest,  perhaps  almost  trivial,  ideas 
which  might  be  a  solution  to  the  problem.  By  under¬ 
standing  why  the  simple  solution  cannot  work,  we 
are  then  ready  to  examine  the  next  simplest  idea. 
Gradually,  we  will  arrive  at  a  more  complex  solution 
which  meets  all  the  requirements  and  which  solves 
the  problem.  And  who  knows,  in  some  cases  the 
simplest  idea  may  even  work! 


Third,  we  must  develop  techniques  for  the  objec¬ 
tive  comparison  and  evaluation  of  systems.  What 
makes  system  A  better  than  system  B?  How  much 
better?  Was  it  worth  the  extra  cost?  Answers  to 
questions  such  as  these  are  important  if  we  hope  to 
build  a  basic  foundation  for  a  science  of  information 
processing.  If  the  operations  analyst  and  the  informa¬ 
tion  system  designer  will  join  hands  to  develop 
criteria  for  handling  these  kinds  of  questions,  this  will 
be  a  major  step  forward  to  improving  our  brainware. 
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INTRODUCTION 

The  development  of  hardware  for  Information  Sys¬ 
tems  has  been  very  slow.  The  first  computers  were 
designed  as  batch  processors  and  now,  approximately 
15  years  and  four  model  changes  later,  the  designers 
know  what  is  required  in  the  way  of  hardware.  In 
the  meantime,  users  have  designed  information 
systems  requiring  far  more  sophisticated  hardware 
design.  It  is  quite  obvious  that  software  is  leading 
hardware  by  at  least  one  model  change.  This  could 
be  the  reason  for  emulators  performing  so  well.  There 
are  growing  indications  that  computer  design  and 
utilization  have  reached  their  infancy. 

Throughout  the  years  thru-put  has  been  used  as 
the  yardstick  for  measuring  computer  performance. 
Perhaps  a  slight  change  to  this  measurement  would 
give  us  a  clearer  picture  of  hardware  capability  and 
performance.  Information  flow  would  be  a  more  ap¬ 
propriate  term  and  would  give  us  a  better  understand¬ 
ing  of  multi-programming,  multi-processing,  batch 
processing,  conversational  and  inquiry  capability. 
In  order  to  give  a  more  vivid  image  of  information 
flow,  a  comparison  to  the  flow  of  liquid  through  a 
pipe  may  best  illustrate  this  point. 

Hatch  processing 

Because  past  emphasis  was  placed  on  batch  pro¬ 
cessing,  the  hardware  design  provides  for  a  better 
information  flow.  Assuming  that  the  internal  memory 
can  be  illustrated  as  a  pipe,  the  cross  sectional  area 
is  proportional  to  the  memory  size.  The  input  and  out¬ 
put  can  be  illustrated  as  fluid  pumps  on  each  end  of 
the  pipe  and  the  processor  itself  can  be  illustrated 
as  a  filter  in  the  pipe  near  the  output  pump.  In  order 
to  process  a  batch  job,  the  input  pump  is  turned  on 
until  the  pipe  is  full  or  all  the  fluid  has  been  pumped 
into  the  pipe.  The  rate  at  which  the  fluid  will  reach  the 
output  pump  is  dependent  upon  the  input  rate  and 
the  efficiency  of  the  filter.  Pipe  lines  are  a  good  illus¬ 
tration  of  this  type  of  processing  because  they  first 


pump  through  one  type  of  fluid  and,  when  all  of  that 
fluid  has  been  pumped  through  the  pump  lines,  they 
then  pump  another  kind  of  fluid  through  this  same 
pipe.  As  can  be  seen,  this  is  the  most  efficient  way  of 
handling  very  large  tasks,  and  it  can  be  seen  that 
many  small  tasks  will  not  be  handled  efficiently  be¬ 
cause  of  the  need  to  clean  all  of  the  fluid  out  of  the 
pipe  before  the  next  task  can  start. 

Multi-programming 

Multi-programming  can  be  considered  as  consist¬ 
ing  of  a  number  of  thin-wall  pipes  inserted  inside  the 
main  pipe  which  is  the  total  internal  memory.  The 
reason  for  using  these  pipes  is  that  the  input  pump 
and/or  the  output  pump  does  not  have  sufficient 
capacity  to  utilize  the  filter  full  time  or  fill  the  main 
pipe.  This  liquid  can  be  used  in  each  of  these  smaller 
pipes.  As  each  smaller  pipe  is  filled,  the  filter  is 
switched  to  that  pipe.  In  this  way  the  filter  is  shared 
amongst  all  the  pipes.  The  space  between  the  pipes 
is  lost  capacity.  In  order  to  help  alleviate  this  waste, 
dynamic  relocation  came  into  being.  The  area  that  is 
lost  due  to  the  thickness  of  the  thin-wall  pipes  is 
executive  overhead*  and  this  varies  from  system  to 
system.  The  pressure  which  must  be  maintained  in 
the  main  pipe  in  order  to  keep  the  smaller  thin-wall 
pipes  from  bursting  is  the  memory  protect  feature  or 
guard  mode. 

Conversational  mode 

The  term  '‘time  sharing"  is  not  used  here  because 
I  am  not  aware  of  any  hardware  system  which  is 
not  used  by  more  than  one  person.  Conversational 
mode  consists  of  as  many  small  pipes  as  possible  in¬ 
serted  in  this  large  main  pipe.  In  order  to  converse, 
the  filter  must  be  readily  available  so  that  it  can  re- 


*  Executive  overhead  is  defined  as  ihe  lime  used  by  lhc  execu¬ 
tive  or  monitor  lo  determine  status  of  various  programs, 
perform  the  necessary  responses  and  pass  control  to  ihe 
proper  program. 
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spond  to  any  input  pump  which  is  started.  At  the 
present  time,  the  input  pumps  have  very  limited  ca¬ 
pacity.  This  is  an  area  in  which  a  great  deal  of  work 
is  required.  Conversational  mode  could  be  considered 
as  an  extension  of  multi-programming.  Multi-pro¬ 
gramming  already  allows  for  interrupt  processing. 
Therefore,  it  appears  that  a  different  compiler  is  all 
that  is  needed  from  a  multi-programming  environment 
to  conversational  mode.  At  the  present  time  there 
is  much  to  be  done  in  the  area  of  designing  hardware 
in  order  to  reduce  the  thickness  of  the  thin-wall 
pipes  (executive  overhead). 

The  conversational  mode  can  be  considered  as  an 
experimental  pipe  line  in  which  the  designer  varies 
the  input  pump  and  observes  the  characteristics  of 
the  fluid  in  the  output  pump  in  order  to  determine 
what  variations  or  modifications  are  required. 

M  ulti-processing 

Multi-processing  provides  more  than  one  filter  in 
the  system.  It  can  be  used  for  back-up  in  case  a 
filter  failure  occurs  ...  it  can  be  used  as  an  augmen¬ 
tation  to  another  filter  in  case  finer  filtering  is  re¬ 
quired  ...  it  can  be  used  in  multi-programming  and 
conversational  mode  where  additional  filters  are  re¬ 
quired  in  order  to  respond  to  the  numerous  pipes. 
Perhaps  the  most  widespread  use  will  occur  when  a 
system  is  used  in  a  combination  of  batch,  multi-pro¬ 
gramming  and  conversational  mode.  The  most  obvious 
use  will  take  place  where  there  is  a  large  number  of 
independent  pipes,  and  these  pipes  can  share  input 
and  output  pumps.  This  is  a  complete  reversal  of 
the  batch-processing  mode.  Perhaps  that  is  why  hard¬ 
ware  designers  are  scratching  their  heads  in  trying 
to  come  up  with  a  good  multi-processing  design.  Let 
us  hope  it  will  not  take  15  years  and  four  model 
changes  to  get  there. 

Compiler  hardware 

Some  very  good  compilers  have  been  written  along 
with  the  not-so-good  and  just  plain  no-good  ones. 
The  very  good  ones  tend  to  be  multi-pass  compilers. 
After  these  many  years  there  is  no  excuse  for  anyone 
to  have  numerous  versions  of  the  same  compiler.  The 
“fast  compile-slow  execute”  and  the  “slow  compile- 
fast  execute”  gimmick  is  the  greatest  hoax  ever  per¬ 
petrated  on  the  computer  users.  This  results  from  a 
definite  lack  of  brainware.  Compilers  require  good 
input  and  output  pumps  along  with  a  rapid  filter 
change.  The  first  pass  requires  a  coarse  filter  to  in¬ 
sure  that  the  filter  does  not  become  clogged  and  im¬ 
pede  a  rapid,  smooth  flow.  On  each  successive  pass, 
a  finer  filter  is  used  until  the  desired  purity  is  achieved. 
The  output  pump  is  connected  to  the  input  pump  dur¬ 
ing  this  cyclic  process.  This  places  a  requirement 


on  the  hardware  for  a  large  rapid-transfer  auxiliary 
memory  .  .  .  one  that  can  keep  the  pipe  filled  to  ca¬ 
pacity  and  both  pumps  working  near  maximum 
capability. 

Hardware  changes 

Each  model  change  has  seen  a  major  change  in 
the  circuitry,  logical  design,  and  manufacturing  pro¬ 
cess  of  the  central  processor.  During  this  same  period 
there  has  been  only  performance  change  in  the  pe¬ 
ripheral  equipment.  Source  data  equipment  has  re¬ 
mained  practically  unaltered.  There  have  been  just 
enough  changes  to  the  central  processor  to  keep  the 
user  excited,  but  when  he  is  finally  able  to  evaluate 
the  capability  of  the  hardware,  he  sees  that  it  falls 
short  of  its  hoped-for  capability.  The  major  change 
for  the  better  has  been  the  obsolescence  of  tape- 
oriented  systems  and  the  emphasis  on  drum-oriented 
systems  with  mass  storage  capability.  Multi-program¬ 
ming  has  existed  for  a  number  of  years  and  seems  to 
have  matured.  The  drum-oriented  systems  have 
proven  themselves  in  satisfying  the  hardware  re¬ 
quirements  for  good  compilers. 

Hardware  expectations 

The  major  new  hardware  improvements  will  be  in 
source  data  readers  and  displays  for  man-machine 
interaction.  Otherwise  there  will  be  a  leveling  off 
of  the  population  of  computer  systems.  It  is  hard  to 
conceive  of  a  great  expansion  in  management  in¬ 
formation  systems  unless  there  is  a  breakthrough  in 
source-data  automation.  The  time  is  rapidly  approach¬ 
ing  when  computer  manufacturers  can  no  longer  rely 
upon  their  circuit  designers  as  the  major  source  of 
processor  improvements.  Major  improvements  in  per¬ 
formance  will  depend  much  more  on  systems  and 
logical  designers.  These  design  engineers  will,  of 
necessity,  become  more  user-oriented.  It  should  be 
easy  for  them  to  become  at  least  as  user-oriented  as 
the  system  programmers.  One  look  at  the  software 
generated  by  the  computer  manufacturers  should 
prove  this  point.  Up-grading  and  enhancing  sections 
of  the  system  without  disturbing  the  rest  of  the  sys¬ 
tem  will  be  the  goals.  Software  will  have  to  be  as 
modular  as  hardware  so  that  as  the  hardware  is 
modified,  the  software  can  be  modified  in  a  parallel 
operation.  We  can  expect  improvements  or  modifica¬ 
tions  of  the  hardware  and  software  as  often  as  every 
year.  Computer  manufacturers  should  be  able  to  re¬ 
spond  to  changing  user  requirements  annually  in¬ 
stead  of  every  four  years  as  they  have  in  the  past. 

SUMMARY 

In  summary,  the  statement  may  be  made  that  hard¬ 
ware  is  that  which  makes  writing  software  nearly 
impossible,  and  if  it  were  not  for  brainware,  it  would 
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be  impossible.  Until  recently  the  hardware  used  in 
information  systems  has  received  more  attention  than 
the  software.  Everyone  seems  to  have  been  enamored 
with  the  words  microsecond  and  nanosecond.  Other 
terms  which  have  intrigued  people  are  real-time  and 
time-sharing.  Yet  the  definition  of  each  term  is  ap¬ 
plicable  to  everything  that  is  done  by  a  computer. 
Computer  hardware  will  have  to  pass  through  another 
generation  in  order  to  reach  maturity.  This  is  evident 
from  the  fact  that  software  and  applications  have  been 
leading  hardware  by  about  three  years.  This  probably 
is  one  of  the  reasons  why  systems  software  is  usually 
so  late.  The  programmers  are  trying  to  provide  soft¬ 


ware  which  is  far  ahead  of  the  hardware  capability. 
This  could  account  for  the  popularity  of  the  IBM-7090 
family.  The  software  was  written  for  the  IBM  704 
and  709  and  therefore  the  next  generation  of  hard¬ 
ware  properly  matched  the  software.  If  this  is  valid, 
why  then  would  wc  have  to  wait  until  1BSYS  13  in 
order  to  get  a  reasonable  operating  system?  Much 
publicizing  has  been  done  about  the  third  generation 
hardware.  Therefore,  the  following  description  may 
be  appropriate:  Third  generation  hardware  is  that 
hardware  which  allows  you  to  use  first  generation 
software  in  order  to  achieve  second  generation  per¬ 
formance. 
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INTRODUCTION 

The  design  and  development  of  Management  Informa¬ 
tion  and  Control  Systems  (MICS)  has  received  con¬ 
siderable  and  evcr-increasing  attention  since  the  intro¬ 
duction  of  general-purpose,  digital,  electronic  com¬ 
puters.  The  stated  purpose  of  all  such  systems  is  to 
provide  management  with  timely,  accurate,  and  per¬ 
tinent  information  at  a  reasonable  cost  so  that  better 
decisions  can  be  made  with  shorter  reaction  times. 
Many  published  studies  indicate  that  these  goals  are 
not  as  readily  achieved  as  would  be  apparent  from  the 
large  number  of  systems  in  existence.  In  many  eases 
they  don’t  even  provide  a  solution  to  the  simpler  prob¬ 
lem  of  Management  Information  Systems  (MIS),  de¬ 
spite  the  many  cliches,  such  as  “Integrated  Systems’’ 
and  “Total  Systems  Approach,”  with  which  the  field 
abounds.  In  fact,  most  such  systems  provide,  at  best, 
only  dated  but  voluminous  outputs. 

This  state  of  affairs  is  better  understood  if  we  exam¬ 
ine  the  software  aspects  of  the  problem  which  result 
from  the  brainware  efforts  expended  thus  far  towards 
its  solution.  Historically  speaking,  all  Management  In¬ 
formation  Systems  thus  far  were  developed  to  eope 
with  specific  problems;  as  a  eonsequence,  many  special- 
purpose  languages  have  come  into  being,  applicable 
only  to  certain  hardware  configurations  and  to  specific 
problem  formulations.  We  can  find  at  least  a  hundred 
separate  installations  using  as  many  different  languages, 
many  of  which  were  heralded  as  the  panacea  to  solve 
all  MIS  or  MICS  problems.  Nevertheless,  a  good  case 
can  be  made  for  management  information  and  control 
systems  designed  to  provide  considerably  more  than 
current  data  and  selected  information.  Such  advanced 
systems  are  not  yet  in  operational  use  because  the  per¬ 
tinent  software  needs  are  not  yet  fully  understood. 
However,  it  is  safe  to  prediet  that  future  EDP  users 
will  indeed  be  able  to  make  intelligent  use  of  intelli¬ 
gence  systems,  to  paraphrase  Norbert  Wiener.  With 
the  advent  of  more  sophisticated  hardware  for  displays 


and  man-machine  communications,  the  long-range  out¬ 
look  for  the  necessary  software  looks  bright  indeed. 
Given  the  appropriate  brainware  support  e.g.,  opera¬ 
tions  research,  such  systems  will  be  able  to  provide 
not  only  factual  data,  but  also  statistical  extrapolations, 
decision  functions,  and  even  the  calculated  risks  asso¬ 
ciated  with  alternative  eourses  of  action  contemplated 
by  management. 

Fundamentals  of  management  information 

and  control  systems 

This  paper  is  concerned  with  the  software  aspects 
of  information  and  eontrol  systems.  In  order  to  attain 
the  necessary  perspective,  we  shall  briefly  examine  the 
framework  within  which  this  software  exists,  that  com¬ 
plex  and  interwoven  pattern  of  hardware,  software, 
and  brainware — not  necessarily  in  that  order.  The 
interplay  between  these  three  elements  cannot  be 
ignored;  it  has  caused  a  continued  increase  in  the  com¬ 
plexity  of  the  hardware,  information  system  languages, 
and  types  of  applications.  As  a  matter  of  fact,  present 
systems  are  mostly  information  systems;  however,  some 
attempts  have  been  made  to  develop  the  eontrol  as¬ 
pects  but  these  efforts  have  not  gone  much  beyond 
the  conceptual  or  experimental  stage. 

Management  information  and  control  systems  are 
concerned  with  the  following: 

•  The  environment  which  is  to  be  monitored  and/ 
or  controlled. 

•  Sensors,  as  well  as  input,  output,  and  display 
devices  to  provide  interface  and  feedback. 

•  Communication  subsystems  which  handle  the 
flow  of  information,  data,  and  eontrol  com¬ 
mands  (if  any). 

•  Loeal  and  central  electronic  computer  subsys¬ 
tems  whieh  process  and  store  information  and 
produce  the  required  outputs. 

The  software  for  management  information  and  eon¬ 
trol  systems,  therefore,  must  handle  with  dispatch  all 
of  the  functions  implied  both  by  the  hardware  and  the 
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intended  application.  This  software  is  in  part  strietly 
hardware-oriented,  differing  from  machine  to  machine 
and  from  system  to  system.  For  example,  executive  or 
operating  systems,  so  necessary  to  efficient  operations, 
fall  into  this  category.  There  will  also  be  software  which 
is  strietly  problem-oriented;  several  programming  lan¬ 
guages  on  various  levels  will  provide  us  here  with  good 
examples.  Finally,  there  will  be  software  which  en¬ 
croaches  on  both  of  these  areas,  such  as  random  access 
file  retrieval  or  eommunieations-oriented  software.  In 
the  subsequent  sections  of  this  paper  we  shall  examine 
critically  the  present  state-of-the-art  of  this  software 
and  we  shall  also  take  a  look  into  what  may  become 
available  in  the  future. 

Information  and  control  system  software 

The  purpose  of  all  information  systems  is  the  collec¬ 
tion,  storage,  retrieval,  and  dissemination  of  timely  in¬ 
formation,  pertinent  to  specific  activities  coming  within 
the  purview  of  management  functions.  All  of  these 
activities  are  best  carried  out  under  control  of  an 
executive  system. 

As  indicated  earlier,  we  shall  separately  consider  the 
software  which  is  essential  to  the  operation  of  such 
systems  and  its  application-oriented  counterpart.  This 
approach  proves  rather  fortuitous  since  the  former  is 
usually  developed  and  supplied  by  the  hardware  manu¬ 
facturer  while  the  latter  originates  at  the  users’  organiza¬ 
tions,  at  least  conceptually. 

Programming  for  specific  applications  is  generally  a 
problem-oriented  task,  thus,  scientific  users  have  al¬ 
ways  been  much  better  off  than  the  system  science 
specialists,  because  of  the  availability  of  FORTRAN 
or  ALGOL.  On  the  other  hand  COBOL  and  its 
esoteric  derivatives  arc  not  yet  fully  developed  or  op¬ 
timized.  There  have  come  into  being  dozens  of  special 
problem-oriented  languages  for  management  informa¬ 
tion  systems — we  shall  give  a  sampling  later.  The 
souree  of  these  languages  ean  often  be  traced  to 
special  hardware  features  available,  or  to  the  need  for 
meeting  special  operational  requirements.  Of  course, 
there  is  also  the  factor  of  human  vanity  associated  with 
the  “invention”  of  a  new  language  and  its  acceptance  on 
a  greater  than  infinitesimal  basis. 

Recently,  a  whole  host  of  new  problems  has  caused 
new  and  special  language  demands,  with  the  introduc¬ 
tion  of  time-sharing  and  its  sophisticated  input/out¬ 
put  deviees.  In  this  connection,  projects  MAC  and 
SKETCHPAD  at  MIT  have  probably  received  the 
greatest  publicity;  similar  work  is  also  going  on  at 
many  other  installations.  The  languages  developed 
under  these  projects  are  intensely  device-oriented  and 
there  appears  little  hope  that  a  Universal  Display  and 
Output  Language  (UDOL — everybody  has  the  right  to 


make  up  acronyms!)  will  be  agreed  upon  or  ean  be 
developed  within  the  foreseeable  future. 

Thus,  programming  for  information  systems  today, 
and  even  tomorrow,  will  be  largely  a  matter  of  using 
what  is  available,  or  of  developing  what  is  needed.  The 
feeling  persists  that  few  programmers  ever  master  any 
of  the  languages  available  to  such  an  extent  that  they 
make  full  use  of  their  total  power.  Rather,  many  pro¬ 
grammers  seem  to  make,  what  appears  at  times,  very 
unreasonable  demands  for  augmentation  or  enhance¬ 
ment  of  existing  languages,  thereby  destroying  their 
universal  and  compatible  character.  This  point  was 
most  recently  under  diseussion  by  H.  Oswald  (1964) 
for  so  well  established  a  language  as  FORTRAN; 
needless  to  say  that  the  picture  is  even  more  ehaotie 
in  the  more  general  case  of  system  seienee  languages. 

Computer  operating  systems 

The  efficient  operation  of  large,  electronic  data  proc¬ 
essing  systems  requires  extensive  and  powerful  execu¬ 
tive  operating  systems.  The  reason  for  this  need  is 
strietly  economical:  best  use  of  the  very  expensive 
hardware  ean  only  be  made  by  holding  human  inter¬ 
vention  to  a  minimum.  The  early  operating  systems 
were  cybernetic  in  character;  they  made  great  demands 
upon  the  ingenuity  of  the  console  operator  and  upon 
the  dexterity  and  skill  of  other  personnel  handling 
tapes  and  cards.  Even  under  the  most  favorable  cir¬ 
cumstances  and  with  highly  trained  personnel,  such 
systems  could  never  exploit  the  full  power  of  the  ma¬ 
chines. 

By  contrast,  the  advent  of  multiprogramming  and 
multiprocessing  computers  has  greatly  changed  this 
picture.  We  have  learned  how  to  organize  modular 
hardware  such  that  all  programs  are  under  eontrol  of 
an  executive  monitor.  The  design  goal  has  been  the 
more  effective  utilization  of  the  hardware;  we  have  also 
learned  that  such  software  systems  were  difficult  to 
construct  unless  the  hardware  had  certain  features  to 
match  the  needs  of  the  monitor  system.  A  good  deal  of 
brainware  has  been  expended  to  make  executive  moni¬ 
tors  as  self-contained  and  as  powerful  as  possible.  Some 
of  the  key  features  that  have  gone  into  their  design 
concern: 

•  Job  schedules .  A  good  monitor  system  must  ree- 
ognize  and  properly  react  to  stated  or  implied 
priorities,  ancillary  device  requirements,  and  many 
other  elements  handled  by  human  dispatchers. 

•  Equipment  allocation.  The  monitor,  especially  in 
the  multi-processing  environment,  must  keep  track 
of  memory  modules  and  peripheral  deviees  and  it 
must  schedule  their  use  in  an  optimum  manner. 

•  Console  communications .  The  monitor  system 
must  provide  the  operator  with  on-line  information 


Software  Considerations 


13 


about  the  state  of  all  jobs  and  about  the  size  of 
the  queues  awaiting  allocation  or  release  of  equip¬ 
ments.  It  must  also  allow  for  human  intervention 
where  desired. 

•  Remote  device  operations.  The  monitor  system 
must  service  all  remote  demands  for  input,  in¬ 
quiry,  printer  or  display  output.  Data  communica¬ 
tion  between  the  central  computer  and  off-site 
peripherals  must  be  handled  exactly  as  if  these 
stations  were  on-site.  This  is  especially  true  for 
time-sharing  and  conversational  operations. 

Multiprogramming  systems 

Efficient  system  operation  implies  the  continuous 
running  of  programs;  a  typical  software  system  which 
was  developed  to  do  exactly  that  is  the  multiprogram¬ 
ming  system  available  for  the  UNI  VAC  1107  machine. 

In  this  system  all  slow-spccd  devices  are  buffered  to 
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the  central  processor  by  way  of  a  large  random  access 
drum.  The  executive  system  is  always  ready  to  serve 
interrupts  from  any  one  of  the  16  input-output  channels 
and  to  go  to  work  for  the  deviee(s)  on  those  channels. 
If  an  on-line  device  requests  attention,  a  symbiont  pro¬ 
gram  facilitates  the  transfer  of  information  between 
that  device  and  its  assigned  drum  buffer  area.  For 
example,  printer  output  from  any  program  is  simply 
transferred  to  the  drum  at  the  time  of  its  generation. 
Later,  the  print  symbiont  program  transfers  the  output 
from  the  drum  to  the  printer,  one  line  image  at  a  time. 
This  transfer  rate  is  governed  by  the  speed  of  the 
printer  and  it  docs  not  interfere  with  the  much  faster 
central  processor  which  carries  on  with  other  work  in 
the  meantime.  In  fact,  this  other  work  may  include 
other  symbiont  programs,  servicing  card  readers  or 
punches,  other  printers,  or  even  remote  devices. 

Figure  1  illustrates  a  typical  day  in  the  life  of  such 
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NOTE:  EACH  CONTINUOUS  LINE  SEGMENT  EQUALS  ONE  COMPUTER  RUN 

Figure  1 


an  operating  system  for  two  machines  “A”  and  “B” 
currently  in  use  at  the  U.  S.  Bureau  of  the  Census. 
Starting  at  0600  hours,  production  runs  of  various 
lengths  were  interspersed  with  test  programs;  a  maxi¬ 
mum  of  nine  such  runs  appear  concurrently  at  1400 
hours.  The  key  to  the  operating  efficiency  of  this  sys¬ 


tem  lies  in  the  fact  that  the  hardware  of  the  machine 
is  capable  of  immediate  response  to  a  large  variety  of 
internal  and  external  interrupts.  Thus  it  appears  to  the 
casual  observer  that  the  on-line  devices  seem  to  operate 
at  full  speed  while  the  central  processor  is  working  at 
full  capacity.  In  reality,  the  on-line  operations  are  not 
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simultaneous  since  the  machine  executes  only  one  in¬ 
ternal  instruction  at  a  time — hence,  the  terminology  of 
concurrent  operations  or  multiprogramming.  However, 
all  of  the  subject  programs,  including  the  symbionts, 
reside  in  main  core  memory  for  the  duration  of  their 
activity  and  the  executive  system  serves  only  to  direct 
the  traffic  and  to  initiate  or  terminate  these  codes  when¬ 
ever  required. 

Multiprocessing  systems 

The  ultimate  in  the  true  time-sharing  approach  is  the 
multi-processing  system.  In  a  typical  such  system,  the 
UN1VAC  1108,  several  large  processors  share  a  large 
core  memory  and  many  high-speed  random  access 
drums  while  control  is  exercised  through  re-entrant 
codes  which  reside  in  core. 

Jobs  are  entered  through  the  card  reader,  from 
magnetic  tape,  or  from  multiplexed  remote  terminals. 
All  incoming  information  is  first  transferred  to  mass 
memory  where  it  enters  a  job  queue.  In  the  case  of 
batch  jobs,  the  system  will  assign  facilities  when  avail¬ 
able,  otherwise  the  job  remains  briefly  in  the  waiting 
line.  Jobs  with  low  priority  improve  their  relative 
position  in  the  queue  simply  by  waiting  it  out;  thus  they 
are  eventually  brought  to  the  attention  of  the  system 
and  do  not  “fall  down  into  the  electronic  eraek.”  The 
system  will  keep  many  jobs  in  core,  dynamically  re¬ 
allocating  memory  space  as  they  terminate;  several  re¬ 
entrant  codes  may  be  in  execution  by  several  processors 
at  the  same  time.  The  codes  for  real-time  jobs  remain 
resident  for  their  entire  duration;  batch  processing 
codes  are  read  in,  executed  and  overlayed  by  other 
batch  codes. 

Time-sharing  of  multiple  processors  is  not  only  more 
economical  but  it  provides  for  greater  resistance  to 
partial  or  temporary  hardware  outages.  For  example,  if 
one  of  the  processors  were  in  maintenance,  the  perform¬ 
ance  of  the  total  system  would  only  be  degraded  but 
service  to  the  user  is  not  interrupted.  The  relevant  soft¬ 
ware  for  such  systems  is  very  complex;  executive  sys¬ 
tems  having  a  hundred  thousand  lines  of  code  on  drum 
and  ten  thousand  lines  permanently  residing  in  core  are 
typical  of  the  effort  required  to  realize  all  the  benefits 
which  can  accrue  through  the  use  of  such  a  system. 

Application  languages 

The  variety  of  languages  (and  their  dialects)  avail¬ 
able  today  for  Management  Information  Systems  is 
illustrated  by  three  representative  examples,  showing 
the  versatility  of  special  programming  languages,  se¬ 
mantic  difficulties,  or  the  foreboding  of  things  to  come, 
as  in  the  case  of  computer  graphics  and  time-sharing. 

A  general  information  processing  system  language 

Information  processing  in  management  information 


systems  implies  the  discovery  and  collection  of  facts, 
their  organization  and  storage  in  a  computer  system, 
their  selective  retrieval  upon  demand,  and  a  synthesis 
process  to  compose  replies  to  queries  and  to  present 
the  output  in  an  appropriate  format.  Such  a  system 
has  been  developed  by  the  Naval  Command  Systems 
Support  Activity  (NAVCOSSACT)  in  Washington; 
similar  systems,  called  ADAM  and  COLINGO,  were 
developed  by  the  MITRE  Corporation. 

A  major  part  of  the  system  generates  and  updates 
data  files.  Reference  is  permitted  to  specific  data  fields 
or  to  logical  pieces  of  data,  allowing  for  operations  on 
the  selected  fields.  The  system  recognizes  fixed  and 
variable  length  items,  both  repeated  and  non-repcated. 
Consecutive  items  may  be  combined  into  sets  and  sets 
may  be  aggregated  into  logical  records.  The  operational 
capability  of  this  file  maintenance  system  is  achieved 
through  the  use  of  macros  which  simulate  a  two- 
address  variable  word  length  computer  whose  memory 
designations  include  master,  transaction,  and  summary 
records.  There  exists  also  a  code  conversion  macro 
LOOKUP  whereby  encoded  and  decoded  data  may  be 
placed  into  or  extracted  from  the  file. 

Another  important  part  of  the  system  deals  with 
information  retrieval.  Inquiries  are  made  by  means  of 
statements  which  resemble  “ordinary”  English.  These 
queries  use  logical  operators  such  as  AND,  OR,  NOT, 
EQ(ual),  N(ot  e)Q(ual),  L(es)S(than) ,  and  macros 
for  FILE,  SECURITY,  TITLE,  SEARCH,  OUTPUT, 
SORT,  and  LIBRARY. 

This  tape-oriented  system  has  been  operational  for 
some  time;  a  random  access  file  version  has  also  been 
developed.  Some  of  the  typical  problems  encountered 
in  the  design  of  this  system  deserve  mention.  Many 
users  expected  that  the  system  would  be  “all  things 
to  all  people”  and  they  were  very  disappointed  when 
one  or  the  other  feature  did  not  meet  their  needs 
exactly.  Certain  users  were  able  to  enforce  their  re¬ 
quests  for  systems  modifications  with  the  result  that 
design  and  development  are  continuing  and  “perma¬ 
nent”  functions.  However,  the  intelligent  use  of  the 
system  was  found  to  depend  mostly  upon  proper 
counselling  of  prospective  users  by  the  operating 
NAVCOSSACT  group. 

The  integrated  date  store  language 

The  engineering  parts  list  problem  deals  with  the 
fact  that  an  original  product  is  a  thing  made  out  of 
things  which,  in  turn,  are  made  out  of  other  things. 
Freguently  hundreds  of  interlocking  parts  lists  arc  need¬ 
ed  to  describe  a  single  item,  such  as  an  electronic  com¬ 
puter.  However,  no  matter  how  involved  it  is,  the 
product  structure  must  be  representable  in  a  com¬ 
puterized  information  system  such  that  simple  parts 
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lists  can  be  extracted  from  it.  Random  access  storage 
is  the  only  known  methodology  whereby  structured 
information  systems  with  more  than  one  dimension  can 
be  developed.  The  key  clement  in  the  design  of  such  a 
system  is  an  incidence  matrix  defining  the  bi-directional 
logical  relation  between  elements  in  the  structured 
lists.  IDS  is  such  a  system,  available  with  General 
Electric’s  large-scale  computers.  Its  basic  file  maintains 
two  chain  structures.  The  first  “Call-Out”  chain  allows 
penetration  of  the  structure  to  the  more  detailed  level; 
it  contains  submaterial  records  from  which  one  can 
proceed  to  the  next  lower  level.  The  second  ^Where- 
Used”  chain  defines  the  upward  relationships  for  a 
material  reeord  and  where  it  is  used  in  the  manufacture 
of  a  higher  level  item. 

The  software  for  this  system  is  an  extension  of 
COBOL.  It  introduces  such  elements  as  “RETRIEVAL 
VIA  CALC  CHAIN”  and  “RANDOMIZE  ON  MA¬ 
TERIAL-1  D.”  Other  typical  commands  such  as  “RE¬ 
TRIEVE  SUBMATERIAL  RECORD”  or  “RE¬ 
PLACE  QUANTITY-REQUIRED  FIELD”  perform 
file  maintenance. 

Experience  with  the  IDS  system  shows  that  it 
eliminates  redundant  data  and  saves  up  to  40  per  cent 
of  the  random  access  file  space  required  by  more  con¬ 
ventional  designs.  In  addition,  operating  times  have 
been  reduced  up  to  50  per  cent  because  efficient  buf¬ 
fering  techniques  and  data  blocking  are  applied  uni¬ 
formly  to  all  IDS  programs. 

Computer  graphics 

On-line  graphical  control  and  display  of  computer- 
processed  information  will  prove  extremely  effective  in 
improving  both  ease  and  speed  of  man-machine  com¬ 
munications  because  of  its  natural  appeal  to  the  human 
mind.  The  most  widely  publicized  programs,  under 
project  SKETCHPAD,  were  developed  at  MIT.  They 
allow  the  user,  among  other  things,  to  draw  into  the 
scope  any  of  three  views  of  a  solid  object  while  the 
computer  prepares  its  digital  representation.  The  re¬ 
quisite  graphical  utility  packages  can  be  linked  with 
other  computer  programs. 

A  list  structure  processing  system  to  perform  such 
tasks  was  developed  at  Lincoln  Laboratory  under  the 
name  of  CORAL  (Class  Oriented  Ring  Association 
Language).  This  language  consists  of  sets  of  operators 
for  building,  modifying,  and  manipulating  list  structures 
which  are  developed  as  rings  in  the  classical  mathe¬ 
matical  sense.  Each  element  in  the  ring  contains  a 
forward  pointer  to  the  next  element.  One  element  in 
each  ring  is  designated  as  the  starting  element  and  all 
other  elements  are  subordinate  to  it.  Subordinate  ele¬ 
ments  contain  a  second  pointer  pointing  backward  to 
the  starting  element.  A  set  of  class  operations  is  avail¬ 


able  with  CORAL.  For  example,  PUT  statements  can 
be  used  to  build  up  rings  until  an  element  is  encoun¬ 
tered  which  belongs  to  two  rings.  Then  a  special  type 
of  connection,  NUB,  is  introduced  to  tie  the  two  rings 
together. 

The  program  has  received  many  interesting  uses. 
For  example,  during  flowchart  programming  the  user 
can  take  a  look  at  his  console  after  calling  up  the  flow¬ 
chart  compiler.  Then  he  ean  construct  global  flowcharts 
on  his  scope  by  calling  up  an  empty  block,  labelling  it, 
and  connecting  it  to  another  block.  Computational  steps 
and  conditional  tests  can  be  entered  via  the  keyboard, 
to  be  displayed  within  the  designated  block.  In  this 
manner  it  is  possible  to  make  a  rough  layout  of  the 
global  flowchart  first  and  then  to  proceed  with  work 
on  the  details  but  always  with  the  whole  context  in 
view. 

A  sampling  of  information  system  languages 

There  does  not  exist  today  a  standardized,  universal 
Information  System  Language  and  it  is  unlikely  that 
such  a  language  will  be  developed  in  the  near  future, 
for  several  reasons.  First,  the  system  concept  is  still 
largely  in  the  development  stage;  and  second,  hardware 
problems  arise  from  mass  storage  requirements,  data 
transmission  needs,  and  central  computer  types.  There 
have  been  as  many  approaches  to  information  system 
languages  as  there  have  been  problem  statements.  The 
approach  in  the  past  has  been  to  develop  systems  and 
languages  to  meet  specific  requirements  and  to  learn 
by  taking  small,  incremental  steps.  The  following  sam¬ 
pling  illustrates  the  variety  of  languages  which  arc  in 
use  today: 

ACSI-MATIC 

This  source  language  is  used  in  an  intelligence  data 
processing  system  for  the  office  of  the  Assistant  Chief 
of  Staff  for  Intelligence.  It  runs  on  the  Sylvania  9400 
computer  using  a  large  Telex  disk  file  for  random  access 
storage  of  data. 

AIRS 

The  Automatic  Information  Retrieval  System  on  the 
IBM  7090  uses  a  keyboard  scheme  to  search  through 
technical  literature. 

ALERT 

The  Automated  Linguistic  Extraction  and  Retrieval 
Technique  is  an  information  handling  program  which 
specifies,  collects,  and  retrieves  large  volumes  of  in¬ 
formation. 

BASEBALL 

This  program  was  written  in  IPL  V  language.  It 
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seans  phrases,  looks  up  words  and  idioms  in  a  stored 
dietionary,  sets  up  a  list  of  attribute-value  pairs,  and 
tallies  the  extracted  data.  It  was  originally  developed 
to  answer  sports  inquiries. 

HAYSTAQ 

This  program  was  written  for  the  SEAC  to  assist 
with  the  seareh  of  ehemical  patent  files  according  to 
specific  topological  and  structural  relationship. 

INFRAL 

This  language  was  developed  at  the  National  Bio¬ 
medical  Rcscareh  Foundation.  It  permits  construction 
of  textual  materials,  such  as  bibliographies  or  ab¬ 
stracts,  from  coordinate  indexed  materials.  This  lan¬ 
guage  is  an  adaptation  of  COBOL  with  ALGOL  type 
statements. 

MOBL 

This  Maero  Oriented  Business  Language  is  used  in 
data  processing  applications  in  conjunction  with  the 
Macro  Instruction  Compiler  Assembler  of  the  SOS 
IBM  7090  system. 

SDI 

This  Selective  Dissemination  of  Information  program 
seans  large  textual  files  and  eomposcs  reports  or  sum¬ 
marizes  information  for  management  use.  The  original 
version  of  this  program  was  written  in  FORTRAN  II. 

WRU 

The  Western  Reserve  University  information  retrieval 
system  converts  textual  information  to  magnetic  tape 
and  searches  this  type  against  encoded  questions. 

VIP 

This  Variable  Information  Processing  system  is  suit¬ 
able  for  small,  nonformalized  files  which  can  be  searched 
in  plain  text  language.  Mnemonic  codes,  abbreviations, 
or  plain  text  language  are  used  in  this  IBM  7090 
program  whieh  was  developed  at  the  Naval  Ordnance 
Labratory,  Corona,  California. 

SUMMARY 

Modern  computer  technology  provides  an  excellent 
basis  for  the  design  of  efficient  information  systems. 
However,  to  exploit  the  full  power  of  the  hardware,  a 
big  investment  in  brainware  and  well-planned  software 
packages  is  needed.  The  latter  cannot  be  developed 
independently  of  the  former  and  the  present  state  of 
the  software  art  shows  elearly  how  much  work  re¬ 
mains  yet  to  be  done.  The  present  emphasis  is  on 
operating  systems  and  special  languages  because  of  the 


great  variety  of  problems  that  have  been  submitted 
for  analysis.  No  universal  language  for  information 
systems  has  been  developed  thus  far  and  it  appears 
doubtful  that  sueh  an  effort  will  be  made  shortly. 
Rather,  many  special  information  system  languages, 
eaeh  more  sophisticated  than  their  predecessors,  will 
be  developed  in  the  years  to  eome.  Hopefully,  this 
trend  will  lead  eventually  to  the  design  of  a  very  general 
information  system  language.  In  the  meantime,  users 
must  learn  to  place  emphasis  on  the  exploitation  of 
existing  software  and  languages  rather  than  to  deelaim 
their  alleged  shortcomings.  A  more  positive  and  foree- 
ful  approach  by  responsible  management  in  this  direc¬ 
tion  would  be  very  beneficial.  As  a  by-produet,  it 
would  also  tend  to  increase  the  intelligent  use  of  our 
not-so-intelligent  but  very  eostly  machines. 
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On  the  role  of  natural  language  in  man-machine  comunication 


by  j.  Bruce  Fraser,  Is/  Lieutenant ,  USAF 

Electronic  Systems  Division,  Air  Force  Systems  Command 
Bedford,  Massachusetts 


INTRODUCTION 

In  a  conference  such  as  this  where  heavy  emphasis  is 
placed  on  the  man-machine  aspect  of  the  use  of  eom- 
putcrs?  the  discussion  very  often  touches  on  the  use  of 
a  natural  language  such  as  English  for  communicating 
with  a  computer-based  system.  And  almost  as  often 
the  comments  heard  arc  that  natural  language  access 
is  too  complicated,  too  inefficient,  and  not  necessary. 
This  may  very  well  be  true.  But  sueh  judgments  arc 
not  usually  based  upon  examination  of  or  experience 
with  a  natural  language  access  capability  but  rather 
on  prejudices  often  held  over  from  school  English 
courses,  misconceptions  born  of  the  ill-eonecived  and 
unsuccessful  attempts  at  mechanical  translation,  and 
a  lack  of  understanding  of  how  sueh  a  capability  might 
be  structured  and  operate.  It  is  my  purpose  in  the 
following  pages  to  sketch  out  the  framework  for  sueh 
a  natural  language  system,  to  indicate  how  it  might 
operate  and  hopefully  thereby  provide  a  better  basis 
for  further  discussion  of  the  merits  of  the  development 
and  subsequent  implementation  of  a  natural  language 
access  capability.  The  system  I  will  discuss  is  hypothe¬ 
tical  though  much  of  the  development  work  has  already 
been  carried  out;  furthermore,  this  organization  is  not 
the  only  tenable  one  at  this  stage  and,  of  course,  not 
necessarily  the  best.  What  I  do  wish  to  emphasize  here, 
however,  is  that  the  role  of  natural  language  in  the 
environment  of  the  digital  computer  seems  very  often 
not  placed  in  the  proper  perspective;  the  following  is 
an  attempt  to  rectify  this  somewhat. 

It  is  clear  at  the  outset  that  the  decision  to  even 
consider  a  natural  language  as  an  access  vehicle  is 
based  on  a  number  of  not  unrelated  factors.  The  type 
of  problem  being  attacked  by  a  user  is  of  paramount 
importance.  Using  English,  one  cannot  expect  to  com¬ 
municate  information  about  second  order  differential 
equations  to  a  machine;  rather  the  language  of  this 
branch  of  mathematics  will  prove  far  more  tractable. 
Similarly,  one  cannot  expect  to  talk  about  the  design 
of  Maeh  2  airfoils  unless  he  uses  a  language  specifical¬ 
ly  tailored  to  graphical  design.  Accordingly,  an  evalu¬ 


ation  of  the  relevance  of  natural  language  should  be 
considered  in  the  context  of  those  problem  areas  which 
involve,  primarily,  symbol  manipulation  with  relatively 
little  computation;  for  examplcj  information  about  in¬ 
ventory,  status  of  forces,  airline  schedules,  and  so  forth. 

A  second  very  crucial  consideration  concerns  the 
type  of  user.  A  frequent  system  user,  especially  one 
who  has  some  familiarity  with  the  operation  of  com¬ 
puter  systems  and  who  is  familiar  with  the  content 
and  structure  of  the  information  stored  in  the  data  base 
of  the  system,  might  be  expected  to  waive  the  op¬ 
portunity  to  use  a  natural  language  and  prefer  a  set 
of  macro  instructions  which  he  has  designed  to  suit 
his  individual  needs.  On  the  other  hand,  the  infrequent 
user  who  knows  very  little  about  the  system  might 
readily  grasp  at  the  opportunity  to  use  his  everyday 
language  to  communicate  with  a  computer  system  which 
would  otherwise  be  beyond  his  ability. 

Still  another  consideration  involves  the  environment 
in  which  the  user  operates.  The  air  traffic  controller  at 
a  large  metropolitan  airport  cannot  afford  the  luxury 
of  typing  into  the  system  a  question  such  as  “What 
holding  patterns  are  now  available?”  when  he  recog¬ 
nizes  that  two  incoming  aircraft  are  following  the  same 
pattern  and  one  must  alter  its  course  immediately. 
However,  the  man  responsible  for  ordering  more  type¬ 
writer  ribbons  for  a  large  organization  might  con¬ 
veniently  query  the  system  with  “How  many  type  43 
carbon  ribbons  do  we  have  left?” 

Finally,  we  can  expect  a  natural  language  system  to 
be  reasonable  only  when  the  user  is  operating  in  an 
on-line  (and  thus  presumably  time-shared)  mode. 
There  seems  to  be  little  advantage  in  using  English  to 
solve  a  problem,  get  an  answer,  or  put  in  some  informa¬ 
tion,  when  the  user  is  required  to  wait  hours  or  even 
days  for  the  result  and  his  next  opportunity  to  pursue 
the  issue  further. 

In  short,  it  seems  reasonable  to  consider  the  use  of 
natural  symbols  for  accessing  a  system  which  manip¬ 
ulates  symbols  (facts,  numbers,  documents)  and 
which  involves  little  numerical  processing  (except,  of 
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course,  some  frequent  computations  preeoded  into 
subroutines  which  could  then  be  called  on),  where 
the  user  is  essentially  a  non-programmer  and  not 
thoroughly  conversant  with  the  system,  where  the 
response  time,  though  important,  is  not  critical, 
and  where  the  user  can  communicate  with  the 
system  in  an  on-line  fashion  using  a  typewriter  and 
perhaps  a  display.  (In  the  subsequent  discussion  we 
will  not  include  the  use  of  a  display  though  it  should 
be  clear  that  it  does  not  significantly  alter  the  sense 
of  the  arguments.)  To  illustrate  such  a  natural  language 
access  capability  we  are  using  a  hypothetical  system 
which  has  in  its  data  base  the  information  contained 
in  the  Official  Airline  Guide ,  that  is,  information  about 
the  schedules  and  accommodations  on  flights  in  the 
continental  United  States.  Wc  will  assume  that  opera¬ 
tion  of  this  system  will  be  on-line  and  that  users  will 
be  secretaries  and  businessmen  interested  in  determin¬ 
ing  available  flights  and  (with  the  requisite  additional 
capabilities)  making  the  appropriate  reservations.  We 
will  first  present  the  organization  of  the  system,  then 
discuss  its  operations  and  indicate  how  the  user  might 
expect  to  ineract  with  it. 

Organization  of  the  system 

We  can  represent  the  organization  of  a  language  ac¬ 
cess  system,  be  it  of  the  artificial  language  or  natural 
language  type,  by  the  schema  in  Figure  1. 


Figure  1 


There  are  essentially  two  components  of  interest 
here:  (1)  the  analysis  component  and  (2)  the  transla¬ 
tion  component.  The  analysis  component  takes  an  in¬ 
put  statement  and  analyzes  it  in  terms  of  the  grammar 
of  the  access  language;  the  result  is  the  linguistic 
analysis  of  the  sentence.  Such  an  analysis  is  a  represen¬ 
tation  of  the  sentence  specifying  (among  other  things) 
syntactic  information  about  the  sentence  such  as  what 
elementary  units  (such  as  words)  the  sentence  is  com¬ 
posed  of,  which  of  these  elementary  unit  sequences 
function  in  this  sentence  as  constituents  in  larger  con¬ 
structions  (such  as  a  noun  being  part  of  a  noun 
phrase),  how  the  constituents  are  combined  in  the  sen¬ 
tence,  and  what  relations  they  bear  to  one  another 
(such  as  a  relative  clause  modifying  a  noun  phrase). 
Figure  2  illustrates  an  oversimplified  syntactic  analysis 
of  the  sentence  “Can  you  fly  from  Boston  to  Chicago?” 


Ideally,  the  linguistic  analysis  of  an  input  sentence 
would  provide  not  only  the  syntactic  analysis  but  also 
the  semantic  interpretation  of  the  sentence.  Thus,  in 
the  example  sentence  the  notion  of  possibility  would  be 
associated  with  the  word  can  since  it  functions  as  an 
auxiliary  verb  (as  opposed  to  when  it  functions  as  a 
noun  as  in  tin  can),  the  notions  of  to  ride  on  an  air¬ 
plane  and  to  glide  through  the  air  under  one’s  own 
power  would  be  associated  with  the  verb  fly  (as  op¬ 
posed  to  when  it  functions  as  a  noun),  and  so  forth. 
There  is,  however,  no  semantic  theory  sufficiently  well 
developed  for  incorporation  into  a  system  such  as  we 
are  discussing;  therefore,  wc  must  be  satisfied  with  a 
syntactic  analysis. 

The  translation  component  has  as  its  input  the  lin¬ 
guistic  analysis  of  the  input  sentence  and  in  terms  of 
this  derives  the  appropriate  set  of  operations  to  be 
earned  out  on  the  data  base.  (These  operations  are 
executed  by  the  data  base  processor  which  is  of  no 
concern  to  us  here.)  Simply  stated,  the  translation  com¬ 
ponent  has  the  task  of  mapping  linguistic  structures  onto 
sets  of  operations  to  be  carried  out  on  data  structures. 
This  requires  that  the  nouns  and  adjectives  of  the  input 
sentence  be  correctly  associated  with  the  appropriate 
rows  and  eolumns  of  the  data  base  files  (or  in  case  the 
information  designated  by  a  noun  or  adjective  involves 
the  result  of  computation  on  the  data  structure,  the 
appropriate  computational  subroutines)  while  the  verb 
of  an  input  sentence  is  mapped  onto  the  appropriate 
operation(s)  on  the  data  structure.  If  the  sentence 
contains  qualifying  adverbial  modifiers  (e.g.  tomorrow 
or  after  4  p.m.)  or  if  a  noun  phrase  is  modified  by  a 
relative  clause  (e.g.  a  flight  which  leaves  Boston  after  1 
p.m.)  the  translation  component  must  recognize  this 
and  determine  the  appropriate  operations. 

To  see  what  is  involved  in  this,  let  us  look  first  at 
the  example  data  base  shown  in  Figure  3.  The  informa¬ 
tion  in  this  example  file  structure  has  been  taken  from 
a  recent  issue  of  the  Official  Air-Line  Guide  and  struc¬ 
tured  in  the  following  way:  each  city  which  is  the 
destination  of  a  scheduled  flight  is  listed  under  the  head¬ 
ing  Destination — Name;  the  time  zone  of  this  city,  the 
limousine  eost  from  the  airport  to  the  center  of  the  city, 
and  a  list  of  the  names  of  the  cities  (Origin — Name) 
from  whieh  flights  to  this  particular  destination  origi- 
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Destination  File  Structure 


Destination 


Name  T-Zone  Lim.jCost  Origins 

Name  F-Zone  hares _ Mights 

D-Time  A-Time  Name  Class  Stops  A/C 


Chicago 

CDT 

$2.00 

Boston 

EDT 

F — 60.85 
Y— 50  85 

New  York 

EDT 

F— 52-30 
Y — 43.00 

New  York 

EDT 

$1.75 

Boston 

EDT 

F— 16.50 
Y— 15.60 

8:00AM 

9:15AM 

AA-57 

F/Y 

0 

B-727 

4:00PM 

5:14PM 

TW-437 

F/Y 

0 

B-727 

5:30PM 

7:50PM 

TW-101 

F/Y 

1 

C-880 

7:00AM 

9:17AM 

UA- 103 

F/Y 

1 

B-707 

7:30AM 

8:27AM 

1  W-42 1 

F/Y 

0 

B-727 

10:00AM 

1  1  :12AM 

AA-251 

F/Y 

0 

C-990 
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nate  are  also  indicated.  Associated  with  each  origin  city 
is  its  time  zone,  the  fare  schedule  (first  class  and  tourist) 
between  it  and  this  particular  destination  city,  and  the 
list  of  scheduled  flights  between  these  two  cities.  Asso¬ 
ciated  with  each  flight  is  its  departure  time,  arrival  time, 
the  airline  name  and  flight  number,  the  class  of  travel, 
the  number  of  stops^  and  the  type  of  aircraft.  Those 
familiar  with  the  actual  Airline  Guide  will  recognize 
that  the  organization  of  the  information  in  Figure  3 
is  not  very  different  from  that  in  the  actual  publication. 

This  data  base,  like  every  other,  consists  of  certain 
pieces  of  information  which  have  been  structured  such 
that  some  of  the  relationships  among  the  pieces  are 
explicit,  some  implicit.  For  example,  while  each  destina¬ 
tion  has  a  time  zone  in  which  it  lies  and  is  the  end 
point  of  a  number  of  flights,  only  the  former  informa¬ 
tion  is  explicitly  stated  in  the  example  destination  file 
structure;  the  latter  information  must  be  obtained  by 
searching  through  the  data  base.  The  particular  data 
base  shown  in  Figure  3  has  been  organized  so  as  to  be 
maximally  efficient  for  obtaining  the  answers  to  a  cer¬ 
tain  class  of  input  questions.  That  is,  there  are  clearly 
some  questions  which  are  simple  to  answer  in  terms  of 
this  data  organization  and  some  which,  though  possible, 
arc  extremely  difficult  to  answer.  To  illustrate  this 
point,  consider  what  is  involved  in  determining  the 
answer  to  the  question,  “Can  I  fly  from  Boston  to 
Chicago?”  To  ascertain  whether  there  is  any  such  flight 
scheduled,  it  is  only  necessary  to  find  Chicago  under 
the  heading  Destination — Name  and  then  look  to  see 
if  Boston  is  listed  under  the  heading,  Origin — Name. 
If  so,  then  the  answer  is  yes;  if  not,  then  the  answer  is 
no.  Similarly,  for  the  question,  “Can  I  fly  from  Boston 
to  New  York  City  nonstop?”  the  answer  can  be  found 
by  simply  locating  New  York  as  the  destination,  Boston 
as  the  origin,  and  checking  if  there  are  any  flights  which 
have  “0”  under  the  heading  Stops. 


But  it  is  simple  to  ask  quite  reasonable  questions 
about  flight  information  the  answers  to  which  cannot 
be  easily  determined.  Even  the  question,  “How  many 
stops  does  American  Airlines  Flight  57  make?”  cannot 
be  answered  immediately  because  the  data  are  not  ar¬ 
ranged  according  to  airlines  and  then  by  flight  numbers 
but  rather  by  destination,  then  by  origin,  then  by  flights, 
then  by  origin,  then  by  flights.  Similarly,  questions 
like,  “What  is  the  quickest  flight  to  take  from  New 
York  to  Los  Angeles?”  “Docs  TWA  or  American 
fly  more  often  between  Reno  and  San  Francisco?” 
“Does  Eastern  fly  to  the  same  cities  as  United?” 
and  “What  is  the  fastest  way  to  fly  from  New 
York  to  Tallahassee,  Florida?”  (there  is  no  direct  flight 
between  these  two  cities)  cannot  be  easily  answered, 
not  because  the  information  is  not  available  in  the  data 
base,  but  because  the  information  is  not  organized  for 
such  questions.  And  there  are  many  reasonable  varia¬ 
tions  on  the  data  base  organization  depending,  of 
course,  on  where  the  emphasis  is  being  placed.  It  is 
quite  conceivable  that  a  travel  agency  would  organize 
the  same  informations  in  an  entirely  different  way  if 
it  were  continually  being  asked  questions  like,  “When 
do  I  have  to  leave  Boston  on  Sunday  in  order  to  make 
a  four  hour  stopover  in  Reno,  yet  be  in  Los  Angeles 
by  midnight?” 

To  see  what  is  involved  in  the  actual  interpretation 
of  an  input  question,  consider  the  sentence  “Can  you 
fly  from  Boston  to  Chicago?”  which  has  the  syntactic 
analysis  shown  in  Figure  2.  It  is  clear  that  the  adverb 
of  origin  from  Boston  modifies  the  verb  fly  and  not, 
for  example,  the  subject  noun  phrase  yon.  The  inter¬ 
pretation  of  the  question  is  not  whether  someone  who 
is  from  Boston  can  fly  to  Chicago  but  whether  it  is 
possible  for  someone  who  happens  to  be  located  in 
Boston  to  fly  between  the  two  cities.  And  in  general, 
the  syntactic  analysis  of  a  sentence  will  represent  those 
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constituents  which  have  a  close  relationship  to  each 
other  ( e.g .  a  relative  clause  modifying  a  noun  phrase 
or  a  time  adverbial  qualifying  the  action  specified  by 
the  verb)  as  being  dominated  by  the  same  constituent. 
In  the  above  sentence,  the  constituent  verb  phrase  (VP) 
dominates  both  the  verb  and  the  two  adverbials.  Of 
course  the  fact  that  two  constituents  are  dominated  by 
the  same  higher  constituent  does  not  imply  that  they 
share  a  particular  type  of  relationship;  for  example, 
the  relationship  between  the  subject  noun  phrase  and 
the  verb  phrase  is  certainly  different  than  that  shared 
between  the  verb  and  the  adverbial  of  origin.  And  it 
turns  out  that  each  occurrence  of  the  same  syntactic 
construction  cannot  be  interpreted  in  the  same  way. 
The  question  “Can  I  return  to  New  York  from  Boston 
after  1 1  p.m.?”  is  ambiguous;  it  can  be  understood  as 
asking  whether  it  is  possible  to  arrive  in  New  York 
after  1 1  p.m.  on  a  flight  originating  in  Boston,  or 
whether  it  is  possible  to  leave  Boston  after  1 1  p.m.  on 
a  flight  heading  for  New  York.  For  the  first  interpreta¬ 
tion,  the  operation  on  the  data  base  would  be  to  deter¬ 
mine  for  Destination — Name  equal  to  New  York  and 
Origin — Name  equal  to  Boston  if  there  is  an  arrival  time 
later  than  11  p.m.;  for  the  second  interpretation,  the 
locations  are  the  same  but  the  departure  time  must 
be  later  than  1 1  p.m. 

Clearly  the  ways  in  which  the  time  adverbial  modi¬ 
fies  the  verb  return  and  its  meaning  have  caused  this 
ambiguity  and  the  difference  of  translation  even  though 
the  syntactic  structure,  as  we  have  analyzed  it,  is  the 
same  for  the  two  cases.  And  as  the  input  sentence  be¬ 
comes  more  complicated  such  as  including  relative 
clauses,  the  translation  becomes  more  complicated  as 
well.  For  a  sentence  such  as  “Can  I  go  to  Chicago  on 
a  flight  which  leaves  Los  Angeles  after  1 1  p.m.?”  the 
translation  component  must  recognize  that  when  a  city 
name  follows  a  verb  like  leave ,  depart ,  go,  take  off,  the 
city  specified  is  to  be  located  in  the  column  containing 
origins  for  a  particular  destination,  in  this  case  Chicago, 
and  that  when  a  time  adverbial  such  as  after  1 1  p.m . 
follows  one  of  these  verbs,  the  column  containing  de¬ 
parture  times  will  be  relevant  rather  than  the  one  con¬ 
taining  arrival  information. 

However,  even  without  a  well  developed  semantic 
theory  and  in  spite  of  the  sorts  of  difficulties  briefly 
touched  on  above,  an  effective  translation  algorithm 
can  certainly  be  designed  for  a  given  data  base  structure 
and  a  given  grammar.  The  number  of  linguistic  types 
(as  opposed  to  tokens)  is  finite  and  in  fact,  as  we  shall 
argue  below,  relatively  small.  Each  type  can,  if  neces¬ 
sary,  be  treated  individually  in  terms  of  the  given  data 
base.  One  interesting  question  is,  of  course,  how  general 
the  translation  can  be  made.  For  example,  the  opera¬ 
tions  related  to  answering  the  question  “Docs  any  air¬ 


line  fly  between  New  York  and  Miami?”  are  almost 
the  same  as  “Does  TWA  fly  between  New  York  and 
Miami?”  the  only  difference  being  that  in  the  second 
case  the  entry  in  the  Name  column  of  the  Flight  in¬ 
formation  is  not  free  but  must  be  TWA.  To  the  extent 
that  this  sort  of  collapsing  of  operations  can  be  ex¬ 
tended,  the  translation  algorithm  can  be  made  more 
efficient.  Similarly,  by  treating  verbs  such  as  arrive, 
land,  come  down,  as  the  same  sort  of  verb,  additional 
simplification  of  the  algorithm  can  be  affected.  How 
compact  the  algorithm  can  be  made  is  certainly  an  open 
question  at  this  point  and  it  depends  to  a  certain  extent 
on  the  class  of  input  sentences. 

But  perhaps  the  more  important  question  is  just  how 
fast  such  natural  language  analysis  and  translation  can 
be  accomplished.  If  the  fairly  superficial  syntactic  anal¬ 
ysis  of  a  sentence  which  is  produced  by  a  syntactic  ana¬ 
lyzer  of  the  Kuno-Oettinger  type  (Kuno,  1967)  provides 
sufficient  information  to  a  translation  component  then 
we  can  expect  sentences  like  “What  flights  which  origin¬ 
ate  in  Boston  go  to  Los  Angeles  after  11  a.m.?”  to  be 
analyzed  in  a  few  seeonds.  If  a  deeper  syntaetican  anal¬ 
ysis  is  required — as  would  appear  to  be  the  case — then 
an  analysis  procedure  of  the  sort  developed  by  Petrick 
(1965)  or  The  MITRE  Corporation  (Zwicky,  et  al., 
1965)  which  handles  a  class  of  transformational 
grammars  will  be  required.  The  above  sentence  taes 
anywhere  from  a  few  seconds  to  a  few  minutes 
when  analyzed  by  these  procedures.  Of  eourse  the 
programming  for  these  systems  was  done  to  maxi¬ 
mize  experimentation  not  speed,  and  one  could  certainly 
anticipate  a  considerable  improvement  in  analysis  time 
in  a  routine  programmed  for  production  work.  Even 
so,  the  time  involved  is  significant.  Furthermore,  no 
estimate  on  the  translation  time  is  available  since  no 
one — to  my  knowledge — has  developed  and  imple¬ 
mented  such  a  capability.  Thus  it  may  well  be  that 
the  amount  of  machine  time  involved  in  processing  a 
natural  language  input  makes  the  system  impractical. 
On  the  other  hand,  what  is  lost  in  machine  time  may 
be  more  than  recovered  in  personnel  efficiency. 

Operation  of  the  System 

On  this  note,  let  us  turn  to  the  role  of  the  user  in 
such  a  system.  Certainly  if  a  natural  language  system 
such  as  this  hypothetical  one  is  to  be  economically 
feasible  the  user  must  not  only  find  the  system  con¬ 
venient  to  use  but  he  must  be  able  to  improve  his  per¬ 
formance  over  that  using  a  faster  query  language-type 
system  or  using  some  eomputerless  approach.  For  it 
is  not  clear  that  from  the  user’s  point  of  view  the 
system  will  appear  all  that  appealing.  There  are  a  num¬ 
ber  of  problems  which  a  user  will  encounter,  some  of 
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which  the  user  of  a  query  language  system  never 
meets.  Let  us  now  examine  some  of  these. 

First  of  all,  it  is  quite  clear  that  no  class  of  users  will 
want  to  use  all  of  English  in  working  in  any  particular 
problem  area  since  not  all  sentence  types  will  apply 
(one  would  not  expect  to  ask  “Can  I  persuade  American 
Airlines  to  fly  me  to  Los  Angeles?”)  and  certainly  only 
a  very  small  portion  of  the  total  vocabulary  of  English 
will  be  relevant.  In  fact,  if  one  begins  asking  questions 
about  some  area  such  as  airline  schedules,  he  finds 
quickly  that,  except  for  exaggerated  paraphrases,  there 
are  relatively  few  ways  to  ask  any  particular  question 
(for  example,  “Can  I  fly  from  New  York  to  Boston?” 
“Is  it  possible  to  fly  from  New  York  to  Boston?”  “Are 
there  any  scheduled  flights  from  New  York  to  Boston?’’ 
or  4  Does  any  airline  fly  from  New  York  to  Boston?”) 
and  that  many  of  the  sentence  types  used  to  ask  for 
one  type  of  information  can  be  used  (with  a  change  of 
vocabulary)  to  ask  for  other  information.  Consequent¬ 
ly,  it  seems  reasonable  to  assume  that  a  quite  “habitable 
language”  (Cf.  Watt,  1966)  can  be  defined  which  does 
not  involve  the  full  range  of  English  sentence  types  but 
rather  a  small  subset  of  them.  But  this  of  course  brings 
up  the  question  of  how  the  user  of  the  system  knows 
what  part  of  English  is  available  to  him. 

The  language  available  to  a  user  cannot,  practically, 
be  presented  to  him  as  a  list  of  the  available  English 
sentence  types  since  for  any  language  consisting  of 
more  than  a  few  dozen  sentences  the  user  cannot  easily 
memorize  this  list  and  will  be  spending  much  of  his 
time  finding  out  what  he  can  say  and  how  he  can  say 
it.  (Furthermore,  although  the  generalization  of  the 
user’s  language  will  be  reflected  in  the  grammar  rules 
which  describe  it,  it  is  extremely  doubtful  that  the 
average  user  could  utilize  these  rules  to  permit  him  to 
recognize  the  acceptable  sentences  of  the  language.) 
Nor  can  the  user  be  presented  with  a  list  of  formats 
into  which  he  can  plug  the  appropriate  names,  attributes, 
and  values — the  technique  used  in  the  query  language 
command  and  control  systems.  Here,  as  above,  this 
approach  involves  a  long  list  of  the  possible  formats 
and  the  possible  vocabulary.  Nor  does  an  on-line  lan¬ 
guage  teaching  capability  seem  feasible  for  a  language 
of  any  complexity.  It  appears  that  what  the  user  for 
a  natural  language  access  system  must  do  is  essentially 
lear  what  restrictions  have  been  imposed  on  the  lan¬ 
guage  that  he  already  knows  (say,  English). 

What,  then,  is  the  sort  of  information  which  the  user 
might  be  given  so  that  he  can  learn  the  limitations  on 
his  language?  First  of  all,  he  might  be  told  the  types  of 
questions  that  can  be  asked  of  the  system.  For  example, 
it  is  not  possible  to  ask  metaquestions  of  the  system, 
questions  such  as  “Can  you  answer  a  question  of  the 
form  .  .  .?”  or  “How  do  I  ask  a  question  about  .  .  .?” 


or  “Do  you  have  information  involving  .  .  .?”  Further¬ 
more,  he  might  be  told  to  avoid  asking  questions  in¬ 
volving  matters  of  evaluation  as  in  “What  would  be  a 
good  flight  to  take  from  Boston  to  Chicago?”  or  “What 
is  the  best  way  to  go  from  New  York  to  Los  Angeles?” 
Here  the  user  is  not  asking  for  factual  information  but 
rather  a  judgment  on  the  part  of  the  system;  but  only 
if  such  a  judgment  capability  is  part  of  the  system,  will 
such  questions  be  included  in  the  access  language. 

The  two  types  of  questions  mentioned  above  deal 
with  the  kind  of  information  required  from  the  sys¬ 
tem.  In  these  eases  the  issue  is  what  is  asked  for,  not 
how  the  question  is  asked.  Elliptical  questions,  those 
questions  requiring  circumstantial  knowledge  of  time, 
place,  etc.,  as  in  “Are  there  any  flights  to  New  York 
on  Friday?”  or  “When  does  the  next  plane  leave  for 
Reno?”  might  pose  one  type  of  problem.  Here  the 
user  would  have  to  know  that  the  machine  would  make 
certain  assumptions  about  the  information  which  is 
lacking  and  what  these  assumptions  could  be.  The 
actual  syntactic  construction  and  the  vocabulary  of  the 
sentence  might  pose  another  difficulty.  The  user  might 
have  to  learn,  for  example,  that  he  could  use  active 
questions  (“Can  I  fly  from  Boston  to  Chicago  on 
TWA?”),  passive  questions  (“Is  the  flight  at  6  p.m. 
from  New  York  to  Miami  operated  by  American  Air¬ 
lines?”),  existential  questions  (“Is  there  a  flight  between 
Los  Angeles  and  San  Francisco  after  11  p.m.?”),  and 
sentences  in  which  an  adverbial  is  questioned  (how 
much?,  how  many?,  how  far?,  how  long?,  where?,  what 
kind  of?,  etc,)  but  not  questions  using  have  as  the  verb 
(“Does  United  Airlines  have  a  flight  from  Washington, 
D.C.  to  Detroit”)  or  querying  maxima  and  minima 
(“What  is  the  fastest  way  to  get  from  New  York  to  Los 
Angeles?”).  Besides  these  syntactic  limitations,  the 
user  will  be  constrained  as  to  vocabulary.  He  might 
have  to  realize,  for  example,  that  the  verbs  fly,  travel, 
and  go  arc  part  of  the  vocabulary  of  the  restricted  lan¬ 
guage  but  that  the  verbs  peregrinate,  trek,  and  cruise 
are  not.  Or  that  airplanes  and  plane  may  be  used  but 
aircraft  may  not.  Or  perhaps  he  will  have  to  know 
that  when  talking  about  a  time  period  he  must  use 
the  word  within  rather  than  in,  as  in  within  two  hours. 
The  point  here  is  that  no  matter  how  extensive  the 
vocabulary  of  the  language  is  made,  there  will  always 
be  words  lying  outside  of  it  and  limitations  on  word 
usage. 

Coupled  with  the  problem  of  a  limited  English 
lexicon  is  the  problem  of  word  meaning.  In  query 
languages,  each  word  has  only  one  meaning.  A  word 
is  treated  unambiguously  by  the  system,  and  the  user 
learns  once  and  for  all  how  the  system  will  interpret 
a  particular  wrord.  Since  there  is  only  a  limited  vocab¬ 
ulary,  memorizing  the  single  meaning  for  each  word 
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is  not  a  difficult  task.  For  the  natural  language  user, 
however,  there  will  be  a  great  tendency  to  use  the  same 
word  in  different  contexts  with  different  interpretations. 
For  example,  the  verb  to  be  conveys  the  notion  of 
equality  in  the  sentence  “Logan  is  an  airport,”  the 
notion  of  attribution  in  “Boston  is  large,”  of  location 
in  “Boston  is  in  Massachusetts,”  of  arriving  in  “The 
flight  will  be  late,”  and  of  remaining  in  “The  plane 
will  be  here  for  four  hours.”  Similar  problems  might 
arise  with  other  verbs  sueh  as  have  and  get  and  with 
many  nouns,  adjectives  and  prepositions  which  have 
more  than  one  meaning  which  can  be  applicable  in 
asking  questions  about  a  particular  set  of  information. 
If  the  system  can  recognize  and  handle  the  various 
meanings,  then  of  course  no  problem  exists;  if,  how¬ 
ever,  the  user  must  be  constrained  to  using  certain 
words  in  one  or  two  speeifie  senses,  then  these  con¬ 
straints  must  be  observed. 

Looking  at  this  problem  of  multiplicity  from  a  slightly 
different  point  of  view,  there  is  the  problem  of  the 
ambiguous  sentence.  We  distinguish  two  different  types 
of  ambiguity:  structural  and  lexical.  The  first  is  charac¬ 
terized  by  the  possibility  of  analyzing  a  sentence  syn¬ 
tactically  in  more  than  one  way.  Thus,  the  sentence 
“Does  TWA  or  Pan  American  fly  from  Boston  to 
Chicago?”  has  both  the  interpretation  “Docs  cither 
TWA  or  Pan  American  or  both  fly  from  Boston  to 
Chicago?”  and  the  interpretation  “Which  airline,  TWA 
or  Pan  American  flies  from  Boston  to  Chicago?” 
Similarly,  the  sentence,  “Will  the  tickets  be  collected 
by  the  stewardess?”  can  be  asking  if  the  stewardess 
will  collect  the  tickets  or  if  someone  will  collect  the 
tickets  over  there  near  (by)  the  stewardess.  The  second 
type  of  ambiguity,  lexical  ambiguity,  is  characterized 
by  a  string  of  words  having  a  single  syntactic  analysis 
but  having  at  least  one  word,  which,  in  terms  of  this 
syntactic  analysis,  has  more  than  one  interpretation. 
The  sentence  “Can  I  fly  on  Eastern  from  Boston  to 
Chicago  after  6  p.m.?”  is  subject  to  the  interpretation 
“Is  there  an  Eastern  flight  operating  between  Boston 
and  Chicago  which  leaves  Boston  after  6  p.m.,”  and 
“Is  there  any  room  on  an  Eastern  flight  operating  after 
6  p.m.,  between  Boston  and  Chicago?”  The  ambiguity 
is  due  in  this  case  to  the  two  interpretations  of  the 
model  verb,  can .  And,  of  course,  there  are  sentences 
like  “Can  I  fly  on  Eastern  from  Boston  to  Chicago 
or  Los  Angeles?”  which  contain  both  types  of  am¬ 
biguity.  If  the  constraints  on  the  language  available 
to  the  user  require  that  all  sentences — at  least  in  the 
sense  that  they  are  relevant  to  the  data  base — be  un¬ 
ambiguous,  then  must  this  be  reflected  in  the  input 
question?  Or  can  sueh  ambiguity  be  tolerated  and 
multiple  answers  provided  with  an  indication  at  the 
way  each  was  arrived  at? 


In  the  foregoing  we  have  mentioned  some  of  the 
problems  which  confront  the  user  of  a  natural  language 
access  system:  the  types  of  questions  he  may  ask,  the 
amount  of  information  he  must  provide  in  the  ques¬ 
tion  he  is  asking,  the  syntax  and  vocabulary  he  may 
use,  and  the  types  of  ambiguities  which  may  arise  when 
he  attempts  to  formulate  an  acceptable  sentence  of 
the  language.  It  may  or  may  not  be  possible  to  require 
the  user  of  such  a  system  to  keep  in  mind  all  of  these 
constraints  on  his  language  when  he  is  attempting  to 
use  the  system.  But  even  if  sueh  awareness  about  the 
subset  of  English  available  could  be  mastered  by  the 
user  it  would  clearly  be  to  the  user's  advantage,  and 
thus  permit  him  to  be  more  efficient,  if  he  could  be 
relieved  of  as  many  of  these  considerations  as  pos¬ 
sible.  What  we  are  suggesting  here  is  that  the  user 
should  not  be  ignorant  of  the  limitations  on  the  ac¬ 
cess  language  but  that  he  would  be  more  efficient  if  he 
could  expeet  assistance  from  the  system  itself  when¬ 
ever  he  makes  a  mistake. 

How  then  could  the  system  assist  the  user  to  formu¬ 
late  a  well-formed  input  question?  (Reeall  that  this 
entire  system  would  be  operating  in  an  on-line,  time- 
shared  mode  and  that  there  would  be  ample  oppor¬ 
tunity  for  convenient  and  frequent  interaction  between 
the  man  and  the  system  he  is  using.)  Probably  the 
simplest  error  is  the  misspelling  of  a  word  or  the  use 
of  a  word  not  included  in  the  lexicon  of  the  language; 
in  either  case  when  the  analysis  routine  begins  scan¬ 
ning  the  sentence  in  order  to  associate  the  possible 
grammatical  categories  with  the  actual  English  words, 
no  entry  would  be  found  for  this  particular  word.  The 
user  could  then  be  immediately  notified  that  such  and 
such  a  word  is  not  currently  in  the  dictionary,  asked 
to  check  the  spelling,  and  if  he  finds  it  to  be  correct, 
to  indicate  synonyms  of  this  word.  Thus,  if  the  user 
had  included  aircraft  in  his  input  question  and  was 
given  this  response,  he  could,  upon  cheeking  the  spelling 
and  deciding  it  was  correct,  indicate  that  aircraft  = 
plane  or  that  aircraft  =  airplane .  Assuming  that  one 
of  these  synonyms  were  in  the  dictionary,  the  analysis 
procedure  would  then  proceed  further.  At  this  same 
point  in  the  analysis  routine,  if  the  user  had  used  a 
word  such  as  have  or  maximum,  or  fastest — words 
which  might  be  in  the  dictionary  but  which  arc  not 
to  be  used  because  they  involve  types  of  syntactic  con¬ 
structions  which  are  not  handled  by  the  grammar — 
the  system  could  make  a  comment  to  the  user,  saying 
in  effect  that  the  use  of  this  paritcular  word  is  pro¬ 
hibited  for  such  and  such  a  reason  and  that  he  should 
rephrase  the  question.  The  user  could,  at  this  point, 
indicate  to  the  system,  that  he  prefers  to  use  certain 
words  in  a  speeifie  way  and  that  the  system  should 
realize  that  when  he  puts  in  a  question.  Thus  each 
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user  could  establish  his  own  dictionary  and  use  the 
words  in  it  as  he  has  indicated.  In  fact,  entire  idioms 
and  abbreviations  could  be  defined  by  a  given  user 
and  the  system  would  interpret  them  appropriately  for 
that  user. 

As  the  analysis  routine  proceeds  to  determine  pos¬ 
sible  syntactic  structures  for  this  particular  sentence, 
it  might  turn  out  that  some  particular  strings  of  words 
remain  unanalyzablc  after  all  possible  analyses  have 
been  attempted.  That  is,  the  vocabulary  was  appropri¬ 
ate  but  the  user  had  formulated  a  sentence  with  a  syn¬ 
tactic  construction  not  handled  by  the  rules  of  the 
grammar  or  this  particular  string  of  words  did  not 
represent  a  grammatical  construction  of  English  at 
all.  In  cither  case,  the  user  could  be  notified  that  the 
string  in  question  is  unanalyzablc  and  be  requested  to 
use  another  construction  in  stating  the  same  question, 
that  is,  provide  a  semantic  paraphrase  of  the  original 
question.  Notice  that  the  same  procedure  which  re¬ 
jects  excluded  syntactic  constructions  would  reject  a 
sentence  in  case  it  contained  a  word  which,  though 
in  the  dictionary,  could  not  be  used  in  that  particular 
construction.  For  example,  if  the  verb  be  is  permitted 
with  noun  phrases  and  adjectives  but  not  preceding 
locative  advcrbials,  then  the  response  to  the  question 
“Is  Logan  Airport  in  Boston?”  could  indicate  that  be 
cannot  precede  a  locative  adverbial.  Here  again,  the 
assistance  to  the  user  does  not  involve  any  great  amount 
of  additional  effort  on  the  part  of  the  system  since  it 
must  do  all  of  this  work  anyway  in  order  to  eventually 
arrive  at  the  linguistic  analysis  (analyses)  of  the  sen¬ 
tence. 

Metaquestions  (described  before)  would  be  rejected 
because  their  syntactic  constructions  are  not  handled  by 
the  rules  of  grammar.  However,  no  distinction  would 
be  made  by  the  grammar  between  this  type  of  question 
and  one  which  has  as  its  goal  the  answer  to  a  question 
from  the  system,  which  question,  if  couched  in  another 
syntactic  pattern  could  be  answered.  Thus,  meta- 
questions  would  be  treated  by  the  analysis  routine  as 
syntactically  ill-formed.  However,  because  the  syntax 
of  metaquestions  is  so  limited  [“Do  (can)  you  answer 
questions  of  the  form  .  .  .?”  “How  do  I  ask  a  ques¬ 
tion  about  .  .  .  ?”  “Do  you  know  (have  information 
[data])  about  .  .  .  ?”]  when  the  analysis  routine  deter¬ 
mines  that  it  is  unable  to  syntactically  analyze  the  in¬ 
put  sentence  the  string  might  be  examined  in  terms  of 
the  metaquestion  syntax  and  the  user  informed  that 
such  a  question  type  has  been  detected  and  is  in¬ 
appropriate. 

If  the  analysis  routine  has  finished  analyzing  the 
input  string  and  has  determined  that  there  exists  a 
structural  ambiguity,  the  user  could  be  notified  that 
such  an  ambiguity  exists  and  then  be  provided  with 


some  or  all  of  the  analyses  of  the  sentence,  so  that  he 
could  indicate  which  interpretation  he  intended.  If  a 
lexical  ambiguity  were  detected,  the  system  could 
simply  indicate  to  the  user  that  some  particular  word 
was  being  used  with  the  following  possible  definitions 
and  request  that  the  user  indicate  which  one  he  in¬ 
tended.  It  should  be  clear  that  only  after  the  linguistic 
analyses  of  the  sentence  have  been  determined  is  it 
reasonable  to  query  the  user  about  lexical  ambiguity. 
Certainly  there  will  be  many  words  in  the  input  sen¬ 
tence  which  have  more  than  one  meaning  but  many 
of  the  irrelevant  meanings  will  be  discarded  during  the 
analysis  procedure.  Thus,  although  the  preposition  in 
can  mean  within  both  spatially  and  temporally,  as  well 
as  at  in  the  sense  of  location,  there  would  be  no  point 
in  stopping  the  analysis  routine  and  asking  the  user 
which  of  the  three  interpretations  of  in  he  intends  in  a 
sentence  like  “When  does  flight  TWA  32  land  in 
Boston?”  since  the  analysis  routine  would  automatically 
determine  that  only  the  locative  interpretation  were 
possible  here.  The  question  of  elliptical  sentences  is 
somewhat  more  difficult  to  resolve  since  in  many  eases 
there  arc  no  linguistic  indications  that  some  pertinent 
information  is  lacking.  For  example,  the  sentence 
“When  docs  the  next  flight  leave  for  New  York?”  is 
perfectly  well  formed  syntactically  and  semantically 
though  to  provide  an  answer  requires  one  to  know  the 
date  and  location  of  the  speaker  as  well  as  flight 
information.  Problems  of  ellipsis  can  only  be  resolved 
in  terms  of  the  information  contained  in  the  data  base 
of  the  system,  its  structure,  and  how  the  linguistic 
analysis  of  the  input  sentence  is  mapped  onto  opera¬ 
tions  on  the  data  base.  However,  when  it  is  known 
what  type  of  information  is  missing,  for  example  the 
location  of  the  speaker,  then  some  standard  assump¬ 
tion  could  always  be  made  and  conveyed  to  the  user. 
If  he  dislikes  the  assumption  as  presented  to  him  he 
could  then  be  free  to  correct  the  machine  and  it  could 
then  proceed  to  determine  the  answer  to  the  user’s 
question. 

What,  then,  is  the  role  that  natural  language  can 
play  as  an  access  language  for  computer-based  sys¬ 
tems?  Wc  have  suggested  that  one  of  the  most  reason¬ 
able  environments  to  place  such  a  capability  would 
be  where  the  infrequent  system  user  interacts  with  the 
system  in  an  on-line  fashion  as  he  requests  pieces  of 
information  from  the  data  base  (wc  have  ignored  the 
question  of  how  this  information  originally  was  entered 
or  how  the  data  base  is  updated).  But  the  processing 
time  for  the  syntactic  analysis  of  an  English  sentence 
is  relatively  long  (at  least  an  order  of  magnitude)  com¬ 
pared  to  the  analysis  of  query  language  systems  such 
as  those  described  in  Barlow  and  Cease  (1965)  and 
Spitzer,  Robinson,  and  Neuse  (1965)  and  the  time  to 
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translate  the  resulting  analysis  into  the  appropriate 
operations  on  the  data  base  of  the  system  eannot  be 
even  estimated  at  this  time  sinee  little  effort  has  been 
expended  in  this  area.  On  the  more  positive  side,  we 
suggested  that  the  availability  of  English  to  the  infre¬ 
quent  user  might  sufficiently  increase  his  effieieney 
over  other  available  methods  to  compensate  for  in¬ 
creased  maehine  time  even  though  he  faees  a  multitude 
of  stumbling  blocks  like  word  usage,  ambiguity>  ac¬ 
ceptable  syntaetie  form  of  sentence,  and  so  forth.  But 
even  if  the  maehine  can  quiekly  deteet  errors  on  the 
user’s  part  and  point  them  out  to  him,  it  is  not  at  all 
elear  that  a  user  will  desire  or  tolerate  such  interaction. 
In  short,  no  definite  conclusions  can  be  drawn  at  this 
time.  The  eoneept  of  using  a  natural  language  to  com¬ 
municate  with  a  maehine  is  eertainly  an  appealing  one 
and  I  feel  that  there  is  at  least  some  possibility  of 
achieving  suceess  in  the  narrow  environment  defined 
at  the  beginning  of  this  paper. 

But  before  any  attempts  are  made  to  design  and 
implement  such  a  system,  I  think  two  efforts  should 
be  undertaken.  First,  the  entire  area  of  translation 
from  linguistic  structure  to  data  structure  should  be 
carefully  studied  to  determine  (among  other  things) 
just  how  detailed  a  linguistic  analysis  is  neeessary, 
whether  translation  need  wait  to  begin  until  the  entire 
linguistic  analysis  is  produced,  what  the  relationship 
is  between  the  type  of  information  requested  and  the 
organization  of  the  data,  what  it  would  entail  to  build 
a  translation  algorithm  which  eould  accommodate 
linguistic  analyses  from  a  elass  of  grammars  (just  as 
Petrick's  (1965)  procedure  ean  analyze  any  sentence  of  a 
language  described  by  a  elass  of  transformational  gram¬ 
mars)  and,  similarly,  accommodate  a  elass  of  data 
structures,  and  so  forth.  Once  this  has  been  done — or 
is  at  least  well  under  way — a  translation  algorithm 
should  be  designed  and  programmed  to  determine  at 
least  the  order  of  magnitude  of  time  involved.  This 
programming  should  be  done  to  maximize  efficiency 


for  clearly  if  translation  time  is  excessive,  sueh  a  system 
has  little  chanee  of  being  economically  practical. 

Second,  an  experiment  should  be  conducted  where 
a  group  of  potential  system  users  are  told  they  have 
such  a  system  at  their  disposal  and  ean  use  the  tele¬ 
type  to  eommunieate.  The  reaction  of  these  users  who 
are  not  actually  receiving  feedback  from  a  natural  lan¬ 
guage  system  but  rather  from  a  combination  of  diagnos¬ 
tic  programs  and  the  experimenters  can  then  be 
studied  as  parameters  sueh  as  language  subsets,  seope 
of  questions^  and  content  of  data  base,  are  altered. 
Certainly  if  sueh  experiments  show  promising  results, 
we  ean  be  guardedly  optimistie.  But  when,  and  only 
when,  both  efforts  show  favorable  results  do  I  think 
we  will  be  in  a  position  to  seriously  consider  design 
of  a  natural  language  aeeess  capability  for  a  eomputer- 
based  system. 

REFERENCES 

1  A.  F.  BARLOW  and  D.  R.  CEASE  Headquarters , 
United  States  Air  Force  Command  and  Control  System 
Query  Language  J.  Spiegel  and  D.  Walker,  Eds. 
Information  System  Sciences:  Proceedings  of  the  Second 
Congress  Spartan  Books,  Inc.,  Washington,  D.  C.  1965 

2  J.  F.  SPITZER,  J.  G.  ROBERTSON,  and  D.  H.  NEUSE 
The  Colingo  System  design  philosophy  J.  Spiegel  and 
D.  E.  Walker  (Eds.)  Information  System  Sciences:  Pro* 
ceedings  of  the  Second  Congress  Spartan  Books,  Inc., 
Washington,  D.  C.  1965 

3  S.  KUNO  Computer  analysis  of  natural  languages  To 
appear  in  Proceedings  of  Symposium  on  Mathematical 
Aspects  of  Computer  Science  New  York  1967 

4  S.  R.  PETRICK  A  recognition  procedure  for  trans¬ 
formational  grammars  Unpublished  M.I.T.  Doctoral 
Dissertation  Cambridge,  Mass.  1965 

5  W.  C.  WATT  Habitability  Technical  Report  Techni¬ 
cal  Operations  Research,  Burlington,  Massachusetts 
May  1966 

6  A.  M.  ZWICKY,  J.  FR1FDMAN,  B.  C.  HALL  and  D.  E. 
WALKER  The  MITRE  syntactic  analysis  procedure  for 
transformational  grammars  AFIPS  Conference  Pro¬ 
ceedings:  Fall  Joint  Computer  Conference  27,  317-326 
1965 


Language  structure  and  graphical  man-machine  communication 


by  William  R.  Sutherland 

Massachusetts  Institute  of  Technology  Lincoln  Laboratory* 
Lexington,  Massachusetts 


Computer  graphics  has  become  an  important  means 
of  man-machine  information  exchange.  Unlike  con¬ 
ventional  computer  languages,  graphical  languages 
have  received  little  study,  and  their  formal  properties 
have  not  been  examined  in  depth.  The  laek  of  precise 
ways  to  formulate  and  represent  graphical  language 
fundamentals  impedes  the  use  of  graphical  techniques 
in  many  problem  areas. 

The  term  “graphical  language”  has  been  used  in  a 
computer  context  with  at  least  two  meanings  (Ross, 
1964;  Teager,  1964).  First,  the  term  has  been  applied 
to  a  light-pen  and  push-button  type  of  language  used 
to  control  an  interactive  system  such  as  Sketchpad 
(Sutherland,  1963)  or  DAC-I  (Jaeks,  1964).  By 
manipulating  the  light-pen  as  a  pencil  and  using  but¬ 
tons  for  commands,  a  user  of  this  kind  of  on-line  lan¬ 
guage  causes  a  pieture  to  appear  on  the  computer’s 
graphic  display.  The  seeond  kind  of  graphical  language 
is  the  pieture  language  of  the  output  itself. 

Use  of  the  light-pen  and  push-button  kind  of  eontrol 
language  circumvents  some  problems  inherent  in  direet 
pictorial  data  input.  Well-known  deficiencies  in  eurrent 
pattern  recognition  techniques  make  it  difficult  to  use 
pictures  as  computer  input  data.  Instead  of  presenting 
the  computer  with  input  data  in  the  form  of  an  already 
existing  pieture,  we  can  provide  explicit  instructions  for 
constructing  the  pieture.  The  extra  information  avail¬ 
able  about  how  the  pieture  was  constructed  makes 
the  computer’s  task  of  interpreting  a  pieture  easier. 
For  example,  the  computer’s  knowledge  that  two  ter¬ 
minals  are  connected  by  a  line  would  be  derived  not 
from  the  presence  of  the  line  in  some  drawing,  but 
from  the  user’s  command,  “CONNECT  A  LINE  BE¬ 
TWEEN  THESE  TWO  TERMINALS.”  Replacing  the 
analysis  of  a  eomplex  situation  by  the  synthesis  of  the 
result  from  simple  components  is  a  well-known  and 
useful  technique. 

Statements  in  either  a  eontrol  language  or  a  pieture 


language  can  be  well-formed  or  ill-formed.  A  typed 
eontrol  statement  like  “DRAW  FROM  (XI,  Yl)  TO 
(X2,  DELETE)”  is  ineorreet  because  the  operation 
DELETE  appears  in  place  of  a  numerical  parameter. 
A  pieture  of  a  flow  ehart  might  be  legal  only  if  a 
continuous  flow  path  exists;  a  missing  flow  eonneetion 
between  two  boxes  would  make  the  flow  ehart  ill- 
formed.  Rules  for  distinguishing  between  legal  and 
illegal  constructions  are  commonly  called  the  syntax 
rules  of  a  language. 

At  present,  we  know  considerably  more  about  the 
syntax  of  eontrol  languages  than  we  do  about  the 
syntax  of  pictures.  A  eontrol  language  is  similar  to  a 
written  language  in  that  it  consists  of  a  linear  sequence 
(in  time)  of  inputs.  The  properties  of  linear  languages 
are  well  understood  and  formalizing  their  syntactic 
rules  is  not  difficult  (Floyd,  1964).  A  pieture  language 
is  two-dimensional,  and  as  yet  we  have  no  general 
method  of  formalizing  its  syntax.  A  number  of  in¬ 
vestigators  are  working  on  the  problem,*  but  to  date 
usable  results  are  not  available. 

Sinee  the  linear  syntax  form  of  a  eontrol  language  is 
well  understood,  there  is  no  theoretical  difficulty  in 
creating  programs  that  will  accept  eontrol  inputs  and 
recognize  their  form.  Several  approaches  to  this  task 
have  been  reported  (Lang,  1965;  Roberts,  1966). 
The  basie  structure  of  a  graphics  system  is  similar  to 
that  of  a  compiler.  Both  accept  a  linear  language  input, 
reeognize  a  construct,  and  take  appropriate  aetions; 
one  ease  creates  machine  code  and  the  other  manip¬ 
ulates  a  pieture.  Once  the  features  of  a  eontrol 
language  are  fixed,  creating  an  input  reeognizer  is  a 
well  specified  task.  The  system  designer  is  faeed  with 
ehoiees  as  to  whieh  operations  to  inelude  in  the  eontrol 
language,  and  how  to  structure  operators  and  their 
parameters.  When  these  decisions  have  been  made, 
known  formalisms  are  adequate  for  describing  the 
eontrol  language  syntax  and  for  creating  the  input 
reeognizer. 


*Operated  with  support  from  the  Advanced  Research  Projects 
Agency. 
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On  the  other  hand,  the  syntactic  properties  of  picture 
languages  pose  certain  difficulties  to  a  graphics  system 
designer.  Two  simple  examples  will  serve  to  illustrate 
how  different  applications  require  different  levels  of 
complexity  in  picture  languages.  First,  consider  an 
interactive  graphics  system  used  as  a  simple  drafting 
assistant.  The  object  here  is  to  use  the  control  language 
for  synthesizing  a  neat  looking  picture  on  the  display. 
Picture  syntax  is  not  relevant  in  this  ease  since  anything 
wc  can  possibly  draw  is  legal.  Second,  consider  a 
system  designed  for  flow  chart  programming.  In  this 
case  the  user  will  draw  a  flow  chart  on  the  computer 
display  and  then  expect  the  computer  to  compile  a 
program  from  the  picture.  The  syntax  of  flow  chart 
pictures  is  relevant  since  the  user  may  request  that 
code  be  produced  from  an  ill-formed  flow  chart.  The 
computer  program  that  compiles  machine  code  from 
a  picture  must  be  able  to  recognize  and  reject  ill- 
formed  flow  charts.  For  both  these  cases  control  lan¬ 
guage  syntax  is  relevant;  the  control  language  must  be 
well  specified  and  contain  operations  suitable  for  draw¬ 
ing  pictures. 

To  determine  that  a  picture  is  syntactically  correct, 
one  might  use  an  analysis  program  that  would  check 
an  entire  picture.  While  such  a  checking  program  is 
feasible  in  many  special  cases,  its  construction  would 
be  assisted  by  a  formal  notation  for  expressing  picture 
syntax.  For  conventional  written  programming  lan¬ 
guages,  one  can  automatically  create  an  input  recognizer 
by  formally  describing  the  language  to  a  “recognizer- 
creating”  program  (Feldman,  1964).  At  present,  there 
is  no  comparable  way  of  describing  and  constructing 
a  recognizer  or  checker  for  pictures. 

Whenever  a  constructive  input  language  is  used,  the 
task  of  checking  a  picture  may  be  combined  with  the 
construction  steps  used  for  creating  the  picture.  The 
tentative  results  of  each  step  may  be  cheeked  for  cor¬ 
rectness  before  being  made  a  permanent  part  of  the 
picture.  Advantages  in  operating  this  way  include  im¬ 
mediate  feedback  on  errors,  and  the  use  of  the  human 
response  time  between  steps  for  picture  error  checking. 
Perhaps  the  most  important  benefit  derived  is  the  re¬ 
duction  of  error  cheeking  into  smaller,  more  easily 
understood  parts.  The  inclusion  of  this  kind  of  step- 
by-step  error  checking  in  a  graphics  system  is  con¬ 
ceptually  simple.  At  the  time  the  actions  in  a  construc¬ 
tion  operation  are  defined  one  must  provide  a  means  for 
determining  that  the  pictorial  result  of  this  new  step 
is  correct. 

Regardless  of  how  picture  checking  is  accomplished, 
however,  one  difficult  part  of  the  system  designer’s 
task  still  remains.  For  each  application  area,  it  is 
necessary  to  provide  detailed  criteria  for  accepting  a 
picture  as  correct.  Any  rules  used  must  be  formulated 


carefully  enough  so  that  one  can  write  a  computer  pro¬ 
gram  embodying  them,  and  for  many  applications  this 
will  not  be  trivial. 

Let  us  assume  that  we  can  create  a  graphics  system 
that  can  determine  whether  a  picture  is  correctly  or 
improperly  constructed.  In  some  applications  this  will 
be  sufficient.  However,  we  would  often  like  to  use  the 
picture  to  represent  and  manipulate  non-pictorial  con¬ 
cepts.  Therefore,  the  computer  must  be  able  to  interpret 
the  meaning  we  ascribe  to  a  well-formed  picture.  One 
way  the  picture  could  be  used  is  as  data  to  a  program; 
a  circuit  diagram  drawn  on  a  scope  could  serve  as  input 
data  to  a  circuit  simulation  program.  The  program 
using  the  picture  as  data  will  determine  the  meaning 
assigned  to  it  by  the  computer.  Pictorial  information, 
for  example  a  flow  chart,  could  also  be  interpreted  as 
a  program.  The  pictorial  procedure  can  in  turn  operate 
on  other  data  which  need  not  be  pictorial.  Thus,  we 
could  draw  a  program  which  could  make  an  on-line 
typewriter  into  a  sophisticated  desk  calculator.  Nu¬ 
merical  answers  would  be  derived  from  typed  inputs 
by  the  drawn  flow  chart  program.  The  flow  chart  is 
only  a  picture,  however,  and  the  meaning  given  to  it 
by  the  computer  is  defined  by  the  programs  which  use 
the  flow  chart  for  instructions. 

In  creating  programs  which  use  a  picture  in  a  more 
than  pictorial  fashion,  a  critical  factor  is  the  system 
designer’s  understanding  of  the  conventions  used  to 
give  meaning  to  a  drawing.  Computer  and  user  must 
share  a  common  understanding  about  the  picture  being 
displayed;  this  can  only  be  accomplished  by  the  system 
designer  who  translates  a  user’s  conventions  into  com¬ 
puter  programs.  People  use  the  basic  conventions  for 
standard  mechanical  drawings  without  much  thought; 
however,  the  task  of  creating  a  system  to  control  a 
machine  tool  from  a  mechanical  drawing  is  non-trivial 
because  as  a  minimum  it  requires  a  thorough  under¬ 
standing  of  the  conventions  used  for  dimension  lines, 
center  line,  auxiliary  views,  cross  sections,  etc.  Similar¬ 
ly,  creating  a  flow  chart  interpreter  or  compiler  requires 
a  careful  analysis  of  flow  chart  conventions.  Before  sys¬ 
tems  can  be  developed  for  many  application  areas,  a 
surprising  amount  of  effort  must  be  devoted  for  formu¬ 
lating  the  details  of  the  kinds  of  drawings  to  be  used. 

Like  natural  written  languages,  natural  picture  lan¬ 
guages  arc  often  imprecise  or  even  inconsistent.  They 
generally  have  developed  without  the  need  for  a  careful 
analysis  of  their  properties  and  characteristics;  people 
just  learn  and  use  them.  We  must  not  expect  picture 
languages  to  be  as  simple  and  tractable  as  standard 
programming  languages.  The  natural  and  intuitive 
features  which  make  pictures  difficult  to  formalize  are 
precisely  those  which  make  them  valuable  for  com¬ 
munication.  The  lack  of  formalisms  for  describing 
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picture  languages  imposes  a  real  burden  on  the  system 
designer  who  must  create  the  programs  for  working 
with  seemingly  inconsistent  pictures.  When  at  last  we 
arc  able  to  state  in  precise  terms  the  meaning  of  a 
picture  and  the  rules  for  forming  definitions  of  correct 
and  incorrect  pictures,  the  task  of  creating  a  graphics 
system  for  any  particular  application  should  be  con¬ 
siderably  simplified. 

REFERENCES 

1  J.  A.  FELDMAN  A  formal  semantics  for  computer 
oriented  languages  PhD  Thesis  Carnegie  Institute  of 
Technology  June  1964 

2  R.  W,  FLOYD  The  syntax  of  programming  languages 
A  survey,  IEFH  Transactions  EC- 13,  346-353  August 
1964 

3  F.  I  JACKS  .J  laboratory  ft>r  the  study  of  graphical 
man-machine  communication  AFIPS  Conference  Pro¬ 
ceedings  1964  Fall  Joint  Computer  Conference,  26 
343-350  Spartan  Books,  Washington,  D.C.  1964 

4  R  A.  KIRSCH  Computer  interpretation  of  English 
text  ami  picture  patterns  IEEF  Transactions  on  Elec¬ 
tronic  Computers  F.C-I3,  363-376  August  1964 

5  L  J  KRAKAU  FIR  .Syntax  and  display  of  printed  for¬ 
mat  mathematical  formulas  MS  I  hesis  MIT  1964 


6  C.  A.  LANG,  R.  B  POLANSKY  and  D.  T  ROSS 
Some  experiments  with  an  algorithmic  graphical  lan¬ 
guage  MIT  Report  ESL-TM-220  August  1965 

7  W  A.  MARTIN  Syntax  and  display  of  mathematical 
expressions  MIT  Project  MAC  Memo  MAC-M-257 
29  uJly  1965 

8  R.  NARASIMHAN  Labelling  schemata  and  syntactic 

descriptions  of  pictures  Information  and  Control  7, 
151-179  1964 

9  R  NARASIMHAN  Syntax  directed  interpretation  of 
classes  of  pictures  Communications  of  the  ACM  9, 
No.  3.  166- 173  March  1966 

10  1.  G.  ROBER  TS  A  graphical  service  system  with  vari¬ 
able  syntax  Communications  of  the  ACM  9,  No.  3, 
173-176  March  1966 

11  D.  T.  ROSS  and  C.  G.  FELDMAN  Verbal  and 
graphical  language  for  the  AFD  system  MIT  Progress 
Report  Project  MAC  Technical  Report  No.  4  May  1964 

12  I  I  SUTHERLAND  Sketchpad:  A  man-machine 

graphical  communication  system  AFIPS  Conference 
Proceedings:  Spring  Joint  Computer  Conference  23, 
329-346  1963 

13  H.  M.  TEAGER  Real  time  man-machine  communica¬ 
tion  with  graphical  languages  First  Congress  on  the  In¬ 
formation  System  Sciences  F.SD-TDR  63-474-5  Janu¬ 
ary  1964 

14  W.  C.  WATT  Morphology  of  the  Nevada  cat- 

tlebrands  and  tlnir  blazons,  port  one  National  Bureau 
of  Standards  Report  No.  9050  10  February  1966 


TACTICAL  COMMAND  AND  CONTROL: 
FIELD  SYSTEMS 


Marlin  G.  Kroger,  Chairman 

Session  Participants: 

Roger  M.  Lilly,  Brigadier  General,  USAF 
Robert  F.  Worley,  Major  General,  USAF 
S.  R.  Brown,  Rear  Admiral,  USN 

Introductory  Remarks 
Marlin  G.  Kroger 

North  Atnerican  Aviation ,  Incorporated 
Anaheim,  California 

Those  of  you  who  attended  the  Tactical  Infor¬ 
mation  Systems  panel  session  at  the  Second  Congress 
on  the  Information  System  Sciences  might  recall 
that  one  of  my  approaches  to  being  a  panel  moderator 
is  to  reverse  the  normal  procedure  of  having  the 
audience  ask  questions  of  the  panel  and,  instead, 
have  the  panel  ask  questions  of  the  audience. 

Teaching  by  asking  questions  is  known  as  the 
Socratic  Method  because  Socrates  used  it  very  effec¬ 
tively,  History  teachers  have  been  known  to  shun 
this  method  of  teaching  because  they  recall  that 
Socrates  died  of  an  overdose  of  hemlock,  presumably 
administered  by  some  of  his  “pupil8"  This  thought 
does  not  bother  me  too  much  because  my  reason  for 
asking  questions  of  the  audience  is  to  get  answers, 
not  to  infuriate  people  by  making  them  think.  In  any 
event,  Information  Sciences  Congress  audience  mem¬ 
bers  obviously  like  to  think. 

Having  thus  reassured  ourselves,  let  ns  consider 
some  questions  that  might  be  appropriate  for  dis¬ 
cussion  in  the  unclassified  environment  of  this 
Congress,  One  such  set  of  questions  relates  to  the 
generation  of  requirements  for  tactical  information 
systems: 

I.  How  can  we  develop  requirements  for  systems 
which  provide  effective  capability  for  self-contained 
tactical  C3  operations  but  which  are  still  consonant 
with  the  trend  toward  centralization  of  command? 
Can  you  suggest  better  methods  than  are  now  em¬ 
ployed?  For  example,  do  the  people  who  understand 
operational  problems  also  have  an  adequate  feel  for 
technical  capability?  Are  they  able  to  obtain  the  re¬ 
sources  necessary  to  find  out  what  they  need? 


2.  Assuming  that  operational  people  in  tactical 
elements  do  isolate  a  problem  that  can  be  solved 
by  the  application  of  technological  ingenuity,  can  they 
persuade  the  service  laboratories  and  system  design 
elements  to  work  on  those  problems?  Is  the  inter¬ 
change  of  information  between  these  groups  ade¬ 
quate?  How  could  it  be  improved? 

3.  What  is  the  role  of  nongovernment  capability, 
profit-seeking  and  nonprofit-seeking,  in  the  develop¬ 
ment  of  requirements?  The  Air  Force  in  its  “L" 
systems  has  made  considerable  use  of  such  groups. 
The  Army  in  its  ADSAF  (Automatic  Data  Systems 
within  the  Army  in  the  Field)  program  is  following  a 
similar  approach.  The  Navy,  on  the  other  hand,  has 
primarily  used  blue-suit  capability  — a  notable 
exception  being  the  ClNCPAC  system.  CINCUS- 
NAVEUR  is  designing  its  automated  operations 
information  system  almost  entirely  with  in-house 
capability.  What  are  the  pros  and  cons  of  this? 

4.  When  the  above  problems  have  been  resolved 
and  a  useful  tactical  system  has  been  designed,  can 
it  be  successfully  procured  in  tactical  quantities  in  a 
timely  fashion  under  present  ASPR  procedures? 
Does  the  two-step  procurement  process  really  work 
and  eliminate  the  low  priced  but  incompetent  bidder? 
Does  the  Contract  Definition  Phase  approach  fit 
tactical  systems  as  well  as  strategic? 

5.  What  about  the  interchange  of  information  be¬ 
tween  tactical  units  of  different  services?  Should  one 
service  take  the  lead  in  developing  systems  for  all 
services  in  certain  tactical  information  system  areas? 
What  is  the  impact  of  international  agreements  such 
as  NATO  standards? 

Another  set  of  questions*  relates  to  problems  with 
current  tactical  command  and  control  systems. 

^Adapted  from  a  previously  unpublished  set  of  questions  pro¬ 
vided  by  H.  L.  Shoemaker  as  a  pari  of  a  taclical  command  and 
conlrol  study  in  which  ihis  author  also  participated. 
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1 .  If  current  programs  are  not  satisfying  the  need 
or  are  not  progressing  rapidly  enough,  what  are  the 
problems? 

a.  Inadequate  technology —  hardware,  software, 
system  design,  reliability,  maintainability, 
weight  and  size? 

Is  it  available? 

Is  it  used  in  current  programs? 

b.  Inadequate  funding?  Service  programming? 
DOD  approval? 

c.  Inadequate  Statement  of  Requirement?  Doc¬ 
trinal  difficulties?  Failure  to  assign  task? 
Coordination  problems? 

d.  Inadequate  planning  or  direction  of  projects? 
Development  competence?  Programming  and 
control?  Technical  direction?  Responsiveness 
to  realistic  needs?  Over-sophistication?  User 
acceptance?  Capability  versus  cost  tradeoff 
studies?  Improper  technical  approach? 

e.  Inadequate  progress?  Poor  execution?  Too 
'‘global”  in  character?  Management  or  approval 
problems? 

f.  Are  major  problem  areas  being  neglected? 
Effects  of  ECM?  Survivable  communications? 
Interfaces  with  other  systems? 

g.  Orientation  for  today’s  operations?  For 
future  anticipated  enemy  situations  or  our  own 
capabilities? 

2.  Will  current  interface  studies  and  standardization 
efforts  achieve  compatibility  of  tactical  command 
and  control  systems? 

a.  Is  compatibility  currently  understood? 

b.  Flow  far  should  standardization  be  carried? 
Commonality  of  equipments?  Electrical  inter¬ 
connection?  Different  standards  to  different 


classes  of  equipment?  Disposal  of  existing 
equipment?  Data  exchange  parameters  (mes¬ 
sage  formats,  report  formats,  common  data 
elements,  data  bases,  computer  programs,  file 
structure)? 

c.  What  are  costs  of  standardization?  What  are 
benefits? 

d.  Are  costs  or  impact  of  standardization  ad¬ 
equately  considered  before  imposition  of 
standards? 

3.  What  about  software  evolution  considerations? 

a.  Can  the  model  O  software  be  expected  to  last 
the  entire  life  of  the  system? 

b.  If  the  answer  to  a.  is  “no,”  how  are  changes  to 
be  controlled?  Completely  by  system  user? 
Flow  should  they  be  justified?  With  tradeoff 
analysis  for  each  evolutionary  change  provided 
to  management  and  funding  agencies  or  by 
arbitrarily  allocated  software  improvement 
budgets? 

c.  How  can  similar  tactical  systems  in  various 
parts  of  the  world  be  kept  similar  enough  and 
compatible  enough  to  support  current  force 
structure  package  concepts?  Should  central 
programming  facilities  be  established  for 
similar  systems  to  provide  updating  and  im¬ 
provement  through  new  programs?  How  can 
this  be  policed?  Can  field  units  be  given  any 
freedom  to  modify  software  and  procedures? 

The  following  two  papers  by  senior  military  per¬ 
sonnel  provide  excellent  insights  into  current  systems 
design  approaches.  Questions  listed  above  for  which 
you  do  not  find  answers  in  these  papers  may  be  con¬ 
sidered  your  homework  assignment,  class. 


The  test  and  evaluation  of  large-scale 
information  processing 
systems  in  the  army 


by  Roger  M.  Lilly,  Brigadier  General ,  USA 
Automatic  Data  Field  Systems  Command 
Fort  Belvoir,  Virginia 


INTRODUCTION 

A  brief  narrative  description  is  presented  of  the  test  and 
evaluation  environment  surrounding  the  Automatic 
Data  Field  Systems  Command  located  at  Fort  Belvoir, 
Virginia.  The  paper  begins  with  a  description  of  the 
organization  and  continues  with  a  discussion  of  those 
areas  which  have  been  determined  as  testing-critieal, 
such  as  retesting  of  software  modifications  and  the 
problem  of  maintaining  external  interfaces  with  the 
other  services  as  their  requirements,  equipments,  and 
systems  continue  to  evolve.  Problems  arc  posed  with 
no  attempt  made  to  delineate  solutions.  The  paper  is 
offered  as  a  means  to  stimulate  discussion  during  the 
Panel  Session. 

ADSAF  and  the  automatic  data  field  systems  command 

Because  of  the  unique  mix  of  doctrine  and  hardware 
involved  in  developing  and  fielding  ADP  systems,  a 
unique  organization,  the  Automatic  Data  Field  Systems 
Command,  was  established  to  earry  out  the  following 
missions:  research,  development  and  implementation  of 
ADP  techniques  and  systems  into  taetieal  units  within 
the  Army  in  the  field  to  assist  in  eommand  and  control 
functions.  These  missions  arc  delineated  in  detail  in 
the  Department  of  the  Army  Implementation  Plan 
‘‘Automatic  Data  Systems  within  the  Army  in  the 
Field  (ADSAF).”  As  the  Commanding  General  of  this 
organization,  whieh  is  a  merger  of  the  Command  and 
Control  Information  Systems  Group  (CCISG)  of  the 
Combat  Developments  Command  and  the  Command 
and  Control  Information  Systems  1970  Project  (CCIS- 
70)  of  the  Army  Materiel  Command,  I  report  directly 
and  concurrently  to  the  Commanding  Generals  of  both 
the  Army  Materiel  Command  and  the  Combat  Develop¬ 
ments  Command.  In  addition,  that  portion  of  my  unit 
whieh  is  in  Germany  is  under  the  operational  eontrol  of 


the  Seventh  U.  S.  Army  for  the  conduct  of  an  experi¬ 
ment  being  eondueted  within  that  unit.  Department  of 
the  Army  Staff  coordination  is  done  directly  through 
the  ADSAF  Management  Office  of  the  Assistant  Chief 
of  Staff  for  Force  Development  (ACSFOR). 

The  Automatic  Data  Field  Systems  Command  is  or¬ 
ganized  functionally  on  a  worldwide  basis.  The  three 
major  systems  under  development  are  the  Taetieal  Fire 
Direction  System  (TACF1RE),  Tactical  Operations  Sys¬ 
tem  (TOS)  and  the  Combat  Service  Support  System 
(CS3). 

TACFIRE  is  designed  to  automate  selected  current 
artillery  functions.  These  inelude,  for  example,  am¬ 
munition  and  fire  unit  status,  fire  planning,  target  in¬ 
telligence,  tactical  fire  control,  technical  fire  eontrol, 
artillery  survey,  and  meteorological  data.  These  func¬ 
tions  have,  for  the  most  part,  been  field  tested,  and  the 
justification — in  terms  of  cost  effectiveness — has  been 
completed.  Fielding  is  scheduled  for  the  1970-73  time 
frame. 

TOS  is  designed  to  assist  in  certain  functions  of 
operations,  intelligence  and  fire  support  coordination. 
Representative  functions  inelude  friendly  unit  status, 
task  organizations,  road  networks,  tactical  troop  move¬ 
ment,  barrier  planning,  radio  frequency  allocation,  and 
engineer  taetieal  operations.  Currently,  the  major  effort 
in  TOS  development  is  centered  with  the  Seventh  U.  S. 
Army  in  Germany.  With  this  development  as  a  basis, 
the  TOS  will  be  subsequently  implemented  Army-wide 
by  the  Automatic  Data  Field  Systems  Command. 

CS3  is  designed  to  assist  in  selected  personnel,  ad¬ 
ministrative,  and  logistical  functions.  Included  in  these 
are  unit  readiness  reporting,  stock  control,  materiel 
readiness  reporting,  ammunition  service,  transportation 
service,  personnel  management,  strength  accounting, 
military  pay,  medical  services,  casualty  reporting,  Mili¬ 
tary  Police  services,  graves  registration,  maintenance 
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services,  and  logistic  administration.  CS3  will  be  tested 
in  the  Continental  United  States  at  Fort  Hood,  Texas, 
and  will  be  implemented  Army-wide  during  the  1970- 
1972  time  frame. 

This,  briefly  is  a  functional  description  of  the  ADSAF 
systems. 

The  Army  test  organization 

The  Automatic  Data  Field  Systems  Command  is  re¬ 
sponsible  for  managing  the  development  of  the  ADSAF 
systems.  The  Electronics  Command  (ECOM)  is  respon¬ 
sible  for  performing  hardware  and  basic  software  ac¬ 
ceptance  and  design  testing,  while  the  Test  and  Evalua¬ 
tion  Command  (TECOM)  is  responsible  for  performing 
all  functional,  user-oriented  system  tests.  Coordination 
for  large-scale  troop  tests  is  normally  obtained  through 
the  Continental  Arms  Command  (CONARC),  which 
supplies  all  troops  for  troop  tests. 

ADSAF  employment ,  scope,  and  interfaces 

Some  of  the  units  in  the  communications  nets  cov¬ 
ered  by  these  three  systems  will  have  only  card  trans¬ 
ceivers  while  others  will  only  have  a  mixed-format 
message  entry  device.  Communication  itself  is  by  radio 
or  wire  over  the  established  communications  nets.  Each 
installation  which  has  a  computer  will  be  dedicated  to 
precisely  one  of  the  three  systems.  Because  of  the  differ¬ 
ing  requirements  of  the  three  systems,  the  types  of 
equipment  which  will  serve  each  will  be  different. 

Though  the  eharaeteristies  will  vary  between  the 
three  kinds  of  equipment,  each  computer  will  be  large- 
scale  and  will  be  surrounded  by  a  significant  number  of 
selected  peripherals,  depending  on  the  functions  and  on 
the  echelon  of  employment.  For  instance,  each  of  the 
three  systems  calls  for  at  least  two  kinds  of  computer: 
a  fast  computer  with  large  immediate  access  and  bulk 
memory  for  rear  echelon  and  large  installation  em¬ 
ployment,  and  a  somewhat  slower  computer  with  smaller 
memory  for  employment  in  more  forward  areas  and 
smaller  installations.  The  forward  areas  have  a  more 
localized  mission  and  require  greater  mobility  of  the 
computer  systems  than  do  the  rear  areas  with  their  large 
over-views  in  mission  and  infrequent  relocations. 

These  pairs  of  configurations  within  each  ADSAF 
system  have  a  requirement  for  direct  eomputcr-to- 
computer  communication.  To  a  lesser  extent,  there  will 
also  be  some  direct  communication  between  the  ADSAF 
systems  themselves.  And,  depending  on  the  local  situa¬ 
tion,  there  will  be  times  when  direct  data  exchange  will 
be  made  with  other  Army  tactical  data  systems  and 
with  the  Air  Force,  Navy,  Marines,  and  sometimes 
NATO.  Interfaces,  therefore,  will  play  a  large  role  in 
the  testing  and  in  the  productive  life  of  ADSAF. 


ADSAF  production! maintenance  philosophy 

The  hardware  portion  of  this  philosophy  is  established 
by  existing  Army  procedures  and  the  ADSAF  plan 
makes  no  radical  changes.  In  essence,  the  total  plan  is: 
one  or  more  contractors  will  produce  the  hardware  and 
the  software  for  the  system  and  will  insure  that  internal 
and  external  interfaces  match  properly.  After  the  hard¬ 
ware  is  in  production  and  field  issue  of  the  hardware- 
software  package  has  begun,  system  maintenance  will 
be  taken  over  by  the  Army.  This  means  both  hardware 
repair  and  software  modifications  and  corrections  will 
be  Army  responsibility.  Certain  kinds  of  hardware 
maintenance  will  be  at  organization  level  with  standard 
support  available  from  depots  in  the  event  of  failures 
requiring  skills  not  found  in  the  organization.  All  soft¬ 
ware  modification  will  in  general  be  by  analysts  and 
programmers  located  at  a  higher  centralized  echelon  in 
an  effort  to  retain  standardization  of  procedures  and 
interfaces  and  reduce  the  requirements  for  specialized 
technical  personnel.  Tests  for  maintainability  will  thus 
be  significant. 

The  systems  will  not  all  reach  the  field  simultaneous¬ 
ly,  but  in  three  steps.  TACF1RE  will  be  fielded  first 
follow  by  TOS  and  CS3,  both  of  which  will  be  under¬ 
going  a  field  concept  test  and  evaluation  during  the 
TACFIRE  development.  This  means  that  the  TACFIRE 
interfaces  will  have  to  be  the  test  standard  for  ADSAF. 
It  also  means  that  simulated  inputs  from  the  other  two 
systems  which  conform  to  these  interfaces  will  have  to 
be  available  during  TACFIRE  testing. 

When  TACFIRE  is  being  tested,  the  majority  of  the 
equipment  undergoing  test  will  be  new  to  the  Army 
inventory.  Software  and  system  testing  of  information 
systems  will  be  relatively  new  to  Army  test  agencies. 

This,  then,  is  a  brief  statement  of  the  ADSAF  en¬ 
vironment  when  the  testing  phase  is  entered. 

Objectives  of  test  and  evaluation;  approaches  for 

attaining  these  objectives 

The  objectives  of  the  test  and  evaluation  of  large- 
scale  information  systems  (or  for  that  matter  any  sys¬ 
tem)  in  the  Army  are  familiar,  i.e.,  to  insure  that  the 
product  obtained  from  development  meets  or  exceeds 
all  applicable  Army  standards  for  quality  of  perform¬ 
ance  in  order  that  only  reliable,  useful,  maintainable, 
cost-effcctivc  and  safe  systems  reach  the  Army  in  the 
Field. 

It  is  currently  envisioned  that  each  new  piece  of 
equipment  and  every  configuration  in  each  system  will 
undergo  the  complete  classical  series  of  tests  from 
Engineer  Design  Tests,  which  are  to  collect  design  data, 
confirm  preliminary  concepts  and  calculations,  and 
determine  compatibility  of  components,  through  Re - 
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search  and  Development  Acceptance  Tests ,  conducted 
by  the  developing  agency  to  insure  that  the  specifica¬ 
tions  of  the  development  contract  have  been  fulfilled, 
to  an  integrated  Engineering  Test/ Service  Test  (ET / 
ST)  on  the  entire  software-hardware  configuration  in 
the  field  under  realistic  physical  and  procedural  en¬ 
vironmental  conditions  under  the  control  of  an  inde¬ 
pendent  test  agency.  This  test  sequence  is  controlled  by 
a  Coordinated  Test  Plan  (CTP)  which  has  as  one 
primary  objective  the  task  of  insuring  that  there  is  no 
duplication  of  tests. 

In  parallel  with  this  sequence  of  test  events,  the  soft¬ 
ware  will  be  undergoing  a  similar,  though  less  well- 
defined  sequence  of  tests.  The  complete  software  pack¬ 
age  must  be  available  for  incorporation  into  the  in¬ 
tegrated  ET/ST.  Likewise,  portions  of  the  software 
paekage  (or  simulation  packages  which  produce  sim¬ 
ilar  functions)  must  be  available  on  a  phased  basis  to 
support  the  hardware  test  sequence. 

The  end  result  of  this  series  of  tests  is  a  so-called 
“type-classified”  system  which  is  ready  for  production 
and  troop  issue.  Items  are  classified  as  one  of  several 
standard  types  when  they  have  been  adopted  as  suitable 
for  Army  use;  when  they  are  acceptable  as  assets  to 
meet  operational  requirements;  are  authorized  for  in¬ 
clusion  in  equipment  authorization  documents;  and  are 
described  in  published  adopted  item  lists.  The  minimum 
standard  acceptable  for  ADSAF  systems  is  Standard  A 
(STD  A):  the  most  advanced  and  satisfactory  items 
currently  available  to  fill  operational  requirements. 
Hopefully,  the  question  of  production-suitability  should 
have  been  resolved  very  early  in  ET/ST  so  that  systems 
can  be  produced  and  ready  for  issue  shortly  after  type- 
classification  is  approved.  Whether  or  not  this  can  be 
done  is  an  open  question  and  is  undergoing  study. 

The  problem  of  software  maintenance 

Because  of  the  complexity  of  these  systems,  the  ET/ 
ST  Sequence  is  expected  to  require  at  least  a  year,  with 
type-classification  coming  a  month  later.  This  is  a  sig¬ 
nificant  amount  of  time,  and  it  becomes  even  more 
significant  if  requirements  are  such  that  software  must 
be  type-classified.  Current  regulations  require  extensive 
retesting  after  each  modification  before  an  item  can  be 
re-type-elassified  through  “Confirmatory”  test.  A  Con¬ 
firmatory  test  (Type  I)  is  a  test  or  investigation  of  a 
system  after  type  classification  as  standard  and  using 
early  production  models,  to  insure  that  required  modifi¬ 
cations  not  previously  tested  are  acceptable  for  issue. 

Now,  ehange  is  the  by-word  of  information  systems. 
The  ADSAF  Systems  arc  to  be  designed  from  the 
outset  with  provisions  for  ease  of  modification  and 
extension.  They  will  be  placed  in  environments  which 


are  characterized  by  change  and  will  by  their  very 
nature  contribute  to  this  change.  It  is  to  be  expected, 
therefore,  that  almost  from  their  first  week  of  employ¬ 
ment,  the  ADSAF  systems  manager  will  begin  to  re¬ 
ceive  requests  for  system  modification.  This  is  no  re¬ 
flection  on  system  design  adequacy  nor  on  user  guid¬ 
ance.  It  is  primarily  a  product  of  long  production  lead- 
times  and  the  progressive  Army  approach  to  incor¬ 
porating  those  procedural  changes  which  arc  required 
to  keep  pace  with  technological  development.  In  short, 
then,  the  ADSAF  systems  will  impact  heavily  on  Army 
procedures,  and  the  functional  software  will,  as  a  result, 
have  to  be  modified  to  reflect  the  changes  produced. 
Each  time  a  change  is  made  there  is  the  possibility  of 
a  new  requirement  being  produced  which  will  create 
the  need  for  an  additional  modification.  This  is  the 
expected  environment.  Of  course  all  modifications  will 
be  analyzed  for  future  impact  and  this  will  be  reflected 
in  the  redesign  in  order  to  reduce  the  number  of  phases 
through  a  time-consuming  test  cycle. 

New  additions  will  also  be  a  consideration,  but  they 
are  a  rather  different  problem.  For  many  of  them  no 
usage  experience  will  be  available  to  temper  design 
plans.  Many  will  require  changes  in  several  portions  of 
the  software  package.  An  example  of  this  is  the  addi¬ 
tion  of  a  new  type  of  round  in  the  ammunition  inven¬ 
tory.  Not  only  a  new  technical  fire  direction  task,  but 
a  new  weapons  effects  and  perhaps  even  a  new  firing 
procedures  package  may  be  required  to  make  the  sys¬ 
tem  fully  responsive  to  the  addition  of  this  round. 

The  problem  then  simply  becomes  the  following: 
how  can  testing  lead-times  be  reduced  to  permit  the 
incorporation  of  software  revisions  and  modifications 
within  a  meaningful  time  after  their  requirement  is 
identified  and  approved?  This  is  almost  equivalent  to 
the  following  question:  does  software  require  type- 
classification?  Or  is  there  a  different  way  of  fielding 
software  with  assurance  of  its  quality,  accuracy,  and 
safety?  Docs  a  Confirmatory  (Type  1)  test  constitute 
an  adequate  test  for  reissue  of  programs  with  modifica¬ 
tions? 

E val nation  criteria 

Of  special  interest  is  the  determination  of  the  actual 
criteria  which  should  be  applied  to  a  software  package 
to  determine  its  efficiency  or  even  its  operational  suit¬ 
ability.  The  testing  of  the  software  portions  of  large- 
scale  electronic  equipment  is  a  relatively  new  experience 
for  the  Army.  There  is  little  backlog  of  experience  in 
this  type  of  testing  on  which  to  draw.  The  TACFIRE 
system  will  be  the  first  large-scale  tactical  data  system 
to  be  tested  by  the  Army,  and,  because  of  this,  great 
pains  will  have  to  be  taken  when  describing  individual 
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test  objectives,  procedures,  data,  evaluation  criteria, 
and  results. 

Of  course,  one  area  of  evaluation  which  is  important 
and  quite  apart  from  testing  is  the  evaluation  of  pro¬ 
curement  prososals.  Here  again  the  Army  is  generally 
inexperienced  though  every  day  this  experience  broad¬ 
ens.  Many  of  the  areas  which  currently  have  relatively 
undefined  value  when  related  to  “efficiency'’  during  pro¬ 
posal  evaluation  will  be  easier  to  describe  precisely, 
weight  properly,  and  evaluate  adequately  after  the  Army 
has  experience  with  fielding  several  large-scale  tactical 
information  systems. 

Environmental  tests 

Such  tests  present  a  peculiar  problem  to  the  testers 
of  the  ADSAF  systems  for  a  number  of  reasons: 

1.  It  is  difficult  to  obtain  and  train  truly  experienced 
personnel  to  operate  a  brand-new  system  during 
an  operational  test. 

2.  It  is  difficult  to  write  a  scenario  which  accurately 
simulates  the  conditions  of  field  use  for  such  a 
large  system. 

3.  Because  of  the  scope  of  each  of  these  systems 
within  the  Army  in  the  Field,  accurate  test  of  the 
system  in  use  would  require  the  services  of  at 
least  a  typc-Corps.  The  requirements  in  men  and 
materiel  for  the  extent  of  service  or  troop  tests 
make  this  somewhat  impractical  and  expensive. 

4.  The  balance  point  between  a  scenario  which  in¬ 
adequately  simulates  actual  system  procedural  en¬ 
vironment  and  a  full-scale  troop  test  is  not  easy 
to  define.  What  confidence  docs  one  have  that 
test  results  obtained  from  a  simulated  test  will 
accurately  reflect  system  behavior  during  troop 
employment? 


5.  What  part  does  partial  system  performance  deg¬ 
radation  play  during  the  testing  cycle?  Howr  is  it 
evaluated? 

Complete  solutions  to  such  problems  arc  not  yet  in 
existence  within  Army  test  agencies.  Until  they  are,  we 
must  be  overly  conservative  and  systems  will  tend  to  be 
over-tested. 

One  area  where  over-testing  is  unlikely  is  the  area 
of  reliability.  When  one  asks  for  a  particularly  large 
MTBF  with  95  per  cent  confidence,  cither  one  has  to 
test  a  single  system  for  a  prohibitively  large  number  of 
hours  or  test  multiple  systems  for  a  correspondingly 
shorter  period  of  time.  Normally  these  tests  occur  rela¬ 
tively  early  in  the  test  cycle  and  hence  in  the  production 
cycle,  so  that  it  is  unlikely  that  more  than  one  system 
will  be  available.  It  is  equally  unlikely  that  sufficient 
hours  of  test  will  be  available  sequentially  to  test  the 
confidence  requirement.  What,  then,  is  the  trade-off? 

There  arc  other  areas  where  complete  testing  prior 
to  fielding  is  impossible.  But  the  Army  wants  assurance 
of  the  safety  margins  of  a  system  before  it  is  fielded. 
These  questions  and  others  like  them  remain  to  be 
answered. 

SUMMARY 

In  this  short  and  essentially  non-tcchnieal  discussion,  I 
have  tried  to  convey  some  idea  of  the  test  and  evalua¬ 
tion  framework  in  which  the  ADSAF  systems  find 
themselves.  Personnel  within  the  Army  arc  seeking,  and 
finding,  answers  to  these  questions  daily,  but  it  is  real¬ 
ized  that  even  when  the  systems  are  fielded,  there  will 
remain  unanswered  questions.  Our  problem  then  be¬ 
comes  that  of  reducing  any  adverse  impact  of  such 
unanswered  questions. 
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The  political  requirement  for  centralized  decision-mak¬ 
ing  at  highest  governmental  levels,  aided  and  abetted  by 
the  remarkable  progress  in  electronics  for  data  process¬ 
ing,  has  reached  all  branches  of  government,  causing 
a  tremendous  amount  of  effort  and  money  to  be  di¬ 
rected  toward  the  refinement  of  information  systems. 
Most  information  systems,  whether  in  industry  or  gov¬ 
ernment,  arc  used  most  of  the  time  for  management’s 
timely  enlightenment  so  that  courses  of  action  called  for 
by  analysis  of  the  data  passed  through  the  system,  may 
be  planned.  Since  information  is  obviously  the  primary 
commodity  in  any  information  system,  the  differences 
between  various  systems  arc  determined  by  what  is  to 
be  done  with  the  information,  where  it  must  flow,  the 
characteristics  of  the  hardware,  the  nature  of  the  soft¬ 
ware,  and  the  environment,  both  physical  and  electronic, 
in  which  it  is  to  function. 

It  is  relatively  simple  to  develop  specifications  for 
fixed  information  systems.  Complexity,  sophistication 
and  information  flow  can  be  established  with  environ¬ 
ment  not  a  serious  consideration.  The  processed  data 
are  used  in  a  fixed  or  specialized  pattern  and  flexibility 
is  not  paramount.  Technical  interfaces  can  be  predeter¬ 
mined  and  communications  lines  do  not  change.  In 
contrast,  few,  if  any,  of  these  static  eharacterietics  arc 
found  in  a  tactical  air  control  system  (TACS).  The  con¬ 
ceptual  and  physical  characteristics  of  a  tactical  com¬ 
mand  and  control  system  differentiate  it  from  any  other 
type  of  information  system.  The  developer  of  a  tactical 
command  and  control  system  quickly  discovers  that 
converting  these  characteristics  into  hard  and  software, 
surfaces  an  almost  infinite  number  of  conflicting  design 
problems.  The  purpose  of  this  paper  is  to  discuss  some 
of  these  design  problems  in  terms  of  conceptual  and 
physical  eharacterietics  of  the  system  and,  using  the 
407L  procurement,  to  point  out  the  role  of  the  user  dur¬ 
ing  the  translation  of  this  design  into  hardware. 

Conceptually,  information  handling  in  any  command 
and  control  system  is  a  closed  loop  involving  a  scries  of 


repeating  steps,  recycled  to  obtain  a  desired  detail  or 
level  of  data.  Steps  in  this  cycle  may  be  described  as 
observation,  interpretation,  integration  and  two-way 
communication  of  information.  This  cycle  forms  the 
nucleus  for  the  functional  attainment  of  command,  com¬ 
munications  and  control  within  a  C3  system.  Command 
assists  the  commander  in  performing  intellectual  tasks 
such  as  memory,  interpretation  of  new  information  with 
respect  to  accumulated  information,  recognition  of  a 
pattern  or  meaning  in  a  complex  of  data,  and  quickly 
projecting  a  course  of  action  as  required  for  the  deci¬ 
sion-making  process.  Inserted  between  command  and 
control  is  the  communications  function.  It  provides  the 
transition  from  one  to  the  other.  Flexible  communica¬ 
tions  enable  the  system  to  impinge  on  the  real  world, 
to  operate  internally,  and  to  connect  with  other  similar 
systems.  The  control  function  allows  the  application  of 
command,  whether  execution  is  centralized  or  decen¬ 
tralized.  Control  is  the  action  interface  with  weapons 
systems.  Even  though  the  C3  system  itself  owns  no 
weapons  and  is  thus  a  distinct  entity  from  other  tactical 
weapons  systems,  the  information  passed  through  it  is 
a  weapon  in  itself,  the  use  of  which  will  make  or  break 
an  operation. 

All  three  of  these  functions  arc  common  to  any  mili¬ 
tary  information  system,  but  the  tactical  system  has  one 
distinctive  requirement  uncommon  to  the  rest:  it  must 
accommodate  these  functions  with  mobile  equipment. 
Its  design  must  be  based  on  the  premise  that  it  will  be 
required  to  operate  anywhere  in  the  world,  under  any 
climatic  conditions,  and  under  various  political  and 
military  constraints.  This  calls  for  the  epitome  of 
mobility  and  flexibility;  it  calls  also  for  maximum  reli¬ 
ability.  These  physical  characteristics  seriously  impact 
upon  the  degree  of  sophistication  and  complexity  per¬ 
missible,  which  in  some  cases  determines  the  quantity 
and  quality  of  information  and  the  rapidity  with  which 
it  can  be  handled.  Thus,  the  balance  between  physical 
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and  conceptual  characteristics  is  much  more  acute  than 
with  a  fixed  information  system. 

Furthermore,  since  the  trend  is  toward  a  higher  and 
higher  level  of  control,  these  field  or  tactical  command 
and  control  systems  must  fit  into  a  national  command 
and  control  environment  and  the  degree  of  sophistica¬ 
tion  must  be  compatible  with  the  conceptual  and  the 
physical  characteristics  of  lateral  and  higher  command 
and  control  systems,  at  least  at  points  of  interface.  The 
mobile  command  and  control  systems  might  be  thought 
of  as  forming  part  of  a  hypothetical  worldwide  network 
of  command  and  control  systems  of  all  types — military, 
political,  economic,  or  what  have  you — all  of  which 
are  tied  together  to  form  a  national  control  system. 
While  it  may  be  argued  that  conceptually  the  entire 
network  would  be  the  same,  the  physical  appearance 
and  environment  of  this  network  changes  as  we  move 
from  the  Washington  level  to  the  jungles  of  Viet  Nam. 
Thus,  conceptual  design  of  a  tactical  command  and 
control  system  centers  about  information  flow  but  in¬ 
cludes  such  other  considerations  as  integration  into  a 
national  system,  interface  with  lateral  systems,  military 
doctrine  and  tactics,  cost  effectiveness,  etc.  Physical  de¬ 
sign  centers  about  the  packaging  of  the  electronics, 
communications,  and  data  processing  into  modular  con¬ 
figurations  that  are  mobile,  both  air  and  surface,  yet 
which  provide  needed  operational  capability.  The  fun¬ 
damental  problem,  then,  in  developing  a  tactical  com¬ 
mand  and  control  system  is  to  determine  the  optimum 
point  between  conceptual  requirements  for  information 
flow,  and  physical  restrictions  dictated  by  mission  en¬ 
vironments,  realizing  that  we  must  often  make  “trade¬ 
offs”  between  the  two  parameters. 

Moreover,  the  physical  and  conceptual  parameters 
whose  balance  we  must  seek  are  not  static  entities.  For 
example,  not  all  conflicts  will  be  like  Viet  Nam.  Not 
all  of  the  physical,  geographic,  and  political  environ¬ 
ments  will  be  like  Viet  Nam.  These  environments,  what¬ 
ever  they  are,  are  not  explicitly  predictable  in  either 
their  combination  or  in  their  impact  on  system  opera¬ 
tion.  Conceptually,  if  it  were  possible  to  have  a  stand¬ 
ard  tactical  commander  and  a  standard  tactical  situa¬ 
tion,  the  definition  of  the  quantitative  and  qualitative 
spectrum  of  information  that  we  need  to  process  would 
be  less  complex.  But  a  cursory  examination  of  tactical 
mission  requirements  makes  it  crystal  clear  that  the 
problem  is  massively  intricate. 

The  tactical  control  system  must  be  dynamic,  adapt¬ 
ing  to  changing  needs.  Conceptually  and  physically, 
it  must  be  capable  of  evolving  with  scientific  advance¬ 
ment  and  it  must  be  more  responsive  than  any  enemy 
system.  Conceptual  development  appears  likely  in  the 
areas  of  symbology,  information  stimulus  and  response, 
information  requirements,  and  programming  of  informa¬ 


tion  flow.  The  trend  in  information-flow  hardware  is 
toward  greater  speed,  more  capacity,  and  sizc/wcight 
reduction.  These  trends,  hopefully,  will  be  accom¬ 
panied  by  increased  reliability. 

Thus,  the  goal  of  overall  system  design  is  a  tactical 
command  and  control  system  composed  of  hardware 
that  is  lightweight,  mobile,  reliable,  and  modular;  that 
will  gather,  process,  communicate  and  display  the 
large  volume  of  information  needed  in  today’s  tactical 
operations;  and  that  is  capable  of  worldwide  operations. 
To  meet  this  goal,  the  designer  of  the  system  must 
thoroughly  understand  the  user’s  requirements  and  the 
user  must  be  cognizant  of  existing  and  predictable 
technical  capabilities. 

The  term  “user”  refers  primarily  to  the  operational 
echelon  which  actually  uses  the  data  produced  by  the 
system.  At  the  top,  the  prime  user  is  the  commander 
of  the  combat  force.  At  lower  echelons,  users  include 
intermediate  commanders,  personnel  who  operate  the 
equipment,  and  large  segments  of  the  commanders’ 
staffs  involved  in  production  and  use  of  the  informa¬ 
tion  processed  by  the  systems.  In  very  broad  terms 
within  the  Air  Force,  the  tactical  command  and  control 
users  arc  the  tactical  operating  commands — TAC, 
PACAF,  and  USAFE. 

It  has  been  a  truism  that  the  user  determines  the 
product;  that  hardware  procurement  begins  with  a 
stated  requirement  from  a  user.  And  this  is  normally 
the  way  a  procurement  cycle  does  commence,  though 
the  user  may  have  been  prompted,  at  least  initially,  by 
guidance  from  a  higher  echelon  or  through  learning  of 
advances  in  the  state-of-the-art  in  industry. 

Historically  wc  can  say  that  whereas  aircraft  weapon 
systems  have  been  designed  primarily  around  machines, 
and  man  adapted  to  them,  a  command  and  control 
system  should  be  designed  around  the  man  and  the 
machinery  adapted  to  him.  Further,  aircraft  specifica¬ 
tions  can  be  expressed  in  such  physical  terms  as 
weight/thrust  ratios  or  energy  maneuver  curves,  but 
such  physical  terms  will  not  adequately  describe  the 
human  engineering  for  tactical  C3  systems.  The  com¬ 
mand  and  control  system  and  its  operator  form  as 
personal  a  man-machine  relationship  as  can  be  imag¬ 
ined.  If  the  user  does  not  participate  during  the  com¬ 
plete  system  development  cycle  or  if  his  participation 
is  sterile  and  artificial,  the  resulting  command  and 
control  product  will  be  simply  a  machine — sterile  and 
artificial — and  not  a  true  command  and  control  system. 
It  is  this  man-machine  relationship,  coupled  with  the 
complex  of  conceptual  and  physical  variables  cited 
above,  which  makes  the  role  of  the  user  so  important  in 
the  design  and  development  of  the  system. 

This  user  role — and  his  participation  with  the  en¬ 
gineers  and  technicians  who  arc  planning  and  develop- 
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ing  new  hardware  for  tactical  command  and  control — 
has  been  repeatedly  illustrated  in  Air  Force  System 
Program  407 L.  The  user  has  been  an  active  member 
of  every  stage  of  this  procurement,  from  the  statement 
of  basic  requirements  to  testing  of  finished  hardware. 
During  this  active  participation,  the  user  and  all  other 
members  of  the  procurement  cycle  have  engaged  in  a 
process  of  continual  refinement  and  clarification  of 
needs  and  technological  feasibility.  The  result  has  been, 
perhaps,  a  new  form  of  symbiosis  between  the  user,  the 
technician,  and  industry  which  may  well  mark  the  path 
for  future  development  of  all  command  and  control 
systems  and  perhaps  other  weapon  systems. 

The  407L  life  cycle  began  with  the  user  production 
of  a  specific  operational  requirement  (SOR)  for  new  C3 
hardware.  This  document,  published  in  September  1964, 
showed  an  early  systemic  approach  to  command  and 
control  by  combining  several  previously  unrelated  hard¬ 
ware  requirements — mobile  command  centers,  light¬ 
weight  radar,  mobile  air  base  equipment,  new  com¬ 
munications  complexes — into  a  single,  definitive  user 
requirement  for  improved  command  and  control  equip¬ 
ment.  Despite  the  inclusiveness  of  the  document,  how¬ 
ever,  there  was  nothing  else  to  indicate  that  this  was 
anything  other  than  a  standard  beginning  for  a  normal 
procurement  action. 

However,  in  February  of  1965,  tactical  C3  users  and 
technicians  participated  in  a  worldwide  advanced  tacti¬ 
cal  air  control  system  study  (AT ACS)  which  would 
form  the  basis  for  many  of  the  more  definitive  hardware 
parameters  for  407 L.  The  study  was  a  user-sponsored, 
engineer-attended,  and  joint  user-technical  product.  It 
represented  an  attempt  early  in  the  system  life  cycle  to 
orient  all  agencies  along  similar  paths.  More  important, 
though,  was  the  tacit  recognition  that  the  user  could  no 
longer  sit  back  and  write  operational  requirements  that 
specify  technical  parameters  too  far  beyond  the  realm 
of  capability,  nor  could  he  state  requirements  that  do 
not  take  full  advantage  of  recent  scientific  change.  For 
many  years,  the  military  requirements  writer  was  not 
responsible  for  precise  statements  of  requirements. 
Today  he  is.  The  AT ACS  study  was  conceived  as  an 
effort  to  satisfy  this  demand  for  precision. 

The  study  relates  Air  Force  doctrine,  operations, 
and  organization  to  tactical  command  and  control  equip¬ 
ment  and  capability,  even  though  the  study  was  pri¬ 
marily  conceptual,  and  dealt  almost  exclusively  in  terms 
of  information  flow.  User  inputs  defined  the  numerous 
terminals  of  the  command  and  control  system  and  de¬ 
tailed  the  information  flow  between  these  terminals  in 
the  form  of  what  data  are  needed,  who  needs  them,  and 
in  what  form.  Perishability  and  priority  of  data  were 
also  considered.  The  question  of  interface  between  the 
TACS  and  external  agencies  served  to  point  out  diffi¬ 


culties  in  defining  system  limits.  This  problem  led  to 
new  definitions  and  changes  in  requirements.  In  several 
areas,  the  study  served  to  identify  new  problems  which 
could  seriousl  yaffect  later  procurement.  Most  impor¬ 
tant,  perhaps,  was  the  user-technician  rapport  estab¬ 
lished  early  in  the  procurement  cycle  by  a  study,  not 
a  part  of  the  procurement  system. 

Programming  of  new  command  and  control  equip¬ 
ment  was  expressed  in  the  proposed  system  package 
plan  (PSPP)  for  407L,  published  in  May  1965,  by  Air 
Force  Systems  Command,  as  a  normal  part  of  the  pro¬ 
curement  cycle.  Closely  related  to  the  AT  ACS  study, 
the  PSPP  discussed  a  system  of  many  pieces  of  hard¬ 
ware,  all  related,  but  which  could  be  procured  through 
different  contracts  from  different  manufacturers  at  dif¬ 
ferent  times.  Thus,  phasing  of  these  procurements  was 
a  most  critical  action.  The  PSPP  was  a  combined  effort 
of  the  technical  agency  Electronic  Systems  Division 
(ESD)  of  Air  Force  Systems  Command  and  the  user. 
Although  it  was  previously  normal  procedure  for  the 
user  to  contribute  most  or  all  of  the  operations  section 
of  such  documents,  user-representatives  participated  in 
the  preparation  of  all  sections  of  this  document — 
overall  programming,  operations,  logistic,  manpower, 
personnel,  budget,  etc.  Thus,  the  user  and  the  engineer 
could  evaluate  mutual  impact  on  the  program  of  such 
factors  as  delivery  schedules  versus  training  require¬ 
ments  and  operational  capability  versus  available  man¬ 
power.  The  result  was  a  schedule  which  realistically 
related  operational  needs,  technical  feasibility,  and  in¬ 
dustrial  capability.  Corporate  preparation  of  the  publi¬ 
cation  rather  than  mere  coordination  after  the  fact 
insured  that  the  technician  understood  operational  re¬ 
quirements  and  the  user  understood  technical  feasibility 
and  cost. 

Detailed  physical  parameters  were  introduced  into 
the  cycle  during  the  preparation  of  various  requests  for 
proposals  (RFP)  to  industry  for  specific  hardware 
items.  Each  RFP  has  been  developed  as  a  joint  user- 
technical  publication,  and  meetings  with  industry  for 
discussions  on  these  proposals  have  been  joint  user- 
technical  meetings. 

Each  formal  guidance  meeting  scheduled  by  ESD 
and  hosted  by  the  participating  industry  was  heavily 
user-attended.  Here,  the  user  was  able  to  give  negative 
guidance  as  to  specific  undesirable  designs  or  ap¬ 
proaches  in  addition  to  positive  guidance  provided  by 
written  specifications. 

All  of  these  interactions,  as  with  the  ATACS  study 
and  the  PSPP,  generated  changes  which  have  been  re¬ 
flected  by  revisions  to  basic  specifications.  For  example, 
initial  requirements  for  a  lightweight  3-D  radar  were 
not  in  consonance  with  available  technology,  i.e.,  the 
specified  range  and  altitude  parameters  could  be  ob- 
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tained  only  in  a  relatively  heavy  set.  However,  trade-off 
between  mobility  and  performance  was  found  feasible 
and  desirable.  In  this  ease,  the  user  preferred  mobility 
over  some  of  the  performance  parameters,  and  a  revised 
set  of  specifications  evolved.  Similar  trade-offs  have 
been  made  in  additional  areas  of  407L  as  mutual  un¬ 
derstanding  of  operational  needs  and  teehnieal  capability 
has  grown.  Usually  the  question  of  ehange  has  revolved 
about  sueh  faetors  as  timeliness  of  delivery  or  sophisti¬ 
cated  performance  versus  weight  or  reliability.  There 
has  been  no  set  pattern  of  responses.  The  user,  however, 
has  been  an  aetive  participant  in  eaeh  decision  and  has, 
on  several  occasions,  revised  his  original  specifications 
to  agree  with  new,  evolved  eoneepts  or  ehanged  tech¬ 
nology. 

The  degree  of  user-partieipation  that  I  have  described 
during  some  of  the  beginning  processes  of  procurement 
has  been  equally  true  for  eaeh  other  step  of  the  system 
life  eyele.  Souree  Selection  Boards  have  been  equally 
user-teehnieal  staffed.  Testing  and  monitoring  of  the 
design  of  all  hardware,  beginning  with  the  first  artiele 
configured  for  inspection  (FACI)  and  extending 
through  Category  1?  II  and  III  testing,  have  and  will 
continue  to  have  detailed  participation  by  the  potential 
user  of  the  system.  In  general,  the  user  has  been  an 
aetive  participant  in  every  step  of  the  procurement 
eyele.  These  interactions  have  generated  changes.  The 
quantity  of  changes — call  it  refinement  of  requirements 
— has  in  itself  been  far  greater  than  usual,  but  beyond 
that  is  the  important  innovation  that  the  user  has  been 
willing  and  able  to  participate  in  eaeh  of  the  mediations 
incidental  to  the  changes. 

Beyond  the  scope  of  the  normal  procurement  cycle, 
several  other  aetions  have  also  played  a  significant 
part  in  407L  C3  procurement  and  further  emphasize  the 
specialized  nature  of  this  C3  life  eyele.  One  sueh  aetion 
is  the  series  of  speeial  meetings,  sponsored  by  the  407L 
Systems  Projeet  Offiee.  These  quarterly  planning  con¬ 
ferences  have  been  attended  by  all  military  ageneies 
that  are  now,  or  will  be  involved  with  the  new  TAGS 
equipment.  Conferees  inelude  representatives  from 
DOD,  USAF,  TAC,  PACAF,  USAFE,  AFSC,  AFCS, 
AFLC,  and  ATC.  The  full  range  of  problems  relating 
to  407L  are  diseussed,  ineluding  programming  of  all 
areas  to  insure  timely  accomplishment  of  production, 
supply  procedures,  personnel  training  and  manning, 
neeessary  organizational  or  operational  or  any  other 
unforeseen  ehanges.  Host  for  the  eonferenee  is  rotated 
among  the  using  commands  and  the  407L  SPO.  These 
meetings  have  served  to  emphasize  the  wide  seope  of 
taetieal  eommand  and  eontrol  and  have  allowed  the 
faee-to-faee  interaction  of  the  many  ageneies  that  are 
involved  in  its  development,  procurement,  testing,  and 
operational  use. 


Another  sueh  action  is  the  change  in  emphasis  on 
taetieal  eommand  and  eontrol  by  the  user,  best  evi¬ 
denced  by  organizational  ehanges  within  the  Taetieal 
Air  Command.  The  number  and  level  of  staff  opera¬ 
tional  personnel  direetly  involved  with  taetieal  eom¬ 
mand  and  eontrol  has  been  dramatically  increased. 
These  operational  personnel  have  been  combined  with 
Air  Foree  communications  and  teehnieal  specialists  into 
a  single  staff  ageney  in  order  to  provide  an  integrated 
and  emphasized  approach  to  taetieal  eommand  and 
eontrol.  Additionally,  a  new  taetieal  eommand  and 
eontrol  unit,  the  602nd  Taetieal  Control  Group,  con¬ 
sisting  of  over  1,400  personnel,  has  been  formed  within 
TAC  substantially  raising  our  capability  in  this  area. 

What  I  have  been  describing  is  the  taetieal  eommand 
and  eontrol  procurement  system  as  it  really  exists. 
There  is  a  new  awareness  among  all  ageneies  that  deal 
with  taetieal  eommand  and  eontrol,  of  speeial  require¬ 
ments  or  procedures  for  its  development.  1  have  em¬ 
phasized  that  the  user  must  understand  the  complete 
procurement  eyele  and  must  participate  in  the  whole 
eyele.  He  must  communicate  with  other  people  who 
work  in  the  eyele.  This  user-partieipation  involves  user- 
obligation  and  expertise.  Other  participating  personnel 
are  similarly  obligated.  The  teehnieian  must  understand 
the  needs  of  the  user;  he  must  insure  that  he  includes 
operational  expertise  as  a  continuous  ingredient  in  the 
amalgam  of  talents  that  contribute  to  the  final  shape 
of  the  system. 

Beyond  this,  the  dynamics  of  our  time  require  con¬ 
tinuous  consideration  of  many  different  variables.  The 
most  obvious  factor  that  impacts  on  the  system  life 
eyele  is  teehnologieal  ehange,  but  other  variables  sueh 
as  eost,  mission  priorities,  political  and  soeial  ehanges, 
and  conditions  of  eonfliet  eombine  and  interact  to 
necessitate  a  continuous  re-evaluation  of  requirements 
and  hardware.  In  a  sense,  our  evolutionary  C3  system 
mutates  even  as  it  is  being  produced,  so  that,  as  mueh 
as  possible,  we  reeeive  operable  hardware  that  is  as 
current  as  the  state-of-the-art  will  allow,  while  eon- 
sidering  sueh  faetors  as  timeliness,  operational  capabil¬ 
ity,  and  eost. 

Our  requirements  statements  must  be  developed  in 
eonsonanee  with  today’s  technology  to  meet  tomorrow’s 
military  need,  and  must  provide  for  evolution  into 
future  systems.  The  answer  to  this  challenge  is  in  a 
system  where  frequent  reviews  of  dynamic  factors  are 
accomplished  and  where  the  system  is  allowed  to  evolve 
while  it  is  being  designed  and  produced.  The  specialized 
eoneeptual  and  physical  charaeteristies  of  a  taetieal 
information  system  eoupled  with  the  peculiar  man- 
maehine  relationship  of  any  true  eommand  and  eontrol 
deviee  make  the  role  of  the  user  dynamie,  unique,  and 
absolutely  essential  during  equipment  development. 
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Session  Summary 

The  “Brooks  BUT  was  passed  by  the  Senate  during 
the  final  days  of  the  first  session  of  the  89th  Congress, 
It  had  been  passed  by  the  House  earlier  in  the  same 
session.  The  Bill  was  signed  by  the  President  during 
the  first  week  of  November  1965  and  became  law 
(P.  L,  89-306). 

This  legislation  provided  specific  authorities  to: 
the  General  Services  Administration  for  the  procure¬ 
ment,  utilization,  and  disposition  of  automatic  data 
processing  equipment,  including  administration  of  an 
ADP  revolving  fund;  the  Department  of  Commerce 
(National  Bureau  of  Standards)  for  the  development 
of  data  processing  standards,  the  conduct  of  research 
in  computer  sciences,  and  provision  of  technical 
assistance  to  Federal  agencies;  the  Federal  agencies 
to  retain  their  responsibility  for  the  determination 
of  their  individual  automatic  data  processing  require¬ 
ments,  including  the  development  of  specifications  for 
and  the  selection  of  the  types  and  configurations  of 
equipment  needed,  and  the  use  to  be  made  of  the 


automatic  data  processing  equipment  and  com¬ 
ponents;  and  the  Bureau  of  the  Budget  for  exercising 
policy  and  fiscal  control  over  the  management  of  the 
Federal  ADP  program. 

The  management  programs  of  the  BOB,  GSA,  and 
NBS  are  being  expanded  and  intensified  to  provide  a 
greater  measure  of  central  policy  direction,  coor¬ 
dination,  and  guidance  to  the  Federal  agencies  in  the 
development  of  computer-based  systems  and  the  ac¬ 
quisition  and  use  of  ADP  equipment,  and  to  provide  a 
more  concentrated  effort  in  dealing  with  Govern¬ 
ment-wide  problems  that  involve  external  relation¬ 
ships  with  the  computer  industry,  American  .Stand¬ 
ards  Association,  and  others. 

The  purpose  of  the  session  is  to  review  the  status 
of  the  above  described  BOB,  GSA  and  NBS  manage¬ 
ment  programs  including  future  expansions  now'  fore¬ 
seen,  together  with  their  impacts  on  the  management 
of  military  ADP  systems. 


43 


COMMAND  SYSTEM  SIMULATION 
AND  DESIGN 


Donald  L.  Drukey,  Chairman 

Session  Participants: 

Thomas  S.  McFee 
Philip  R.  Vance 
Andrew  E.  Wessel 


Introductory  Remarks 
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By  way  of  introducing  our  four  papers,  I  would  like 
to  give  some  comments  that  I  had  planned  to  make  as 
introductory  remarks  had  the  Third  Congress  been 
held. 

First,  I  believe  that  our  authors  share,  perhaps 
more  than  1  had  hoped  at  the  time  we  established  our 
group  of  experts,  in  our  opinions  as  to  how  much  can 
be  done  and  where.  1  think  that  we  are  all  indebted 
to  Tom  McFee  for  the  phrase,  “Command  Support 
Systems,”  to  describe  what  we  are  really  planning  to 
discuss.  Let  me  outline  my  personal  position  as  to 
what  I  believe  we  can  agree  on  with  respect  to  that 
portion  of  a  Command  Support  System  which  can 
be  designed  and  implemented  outside  the  framework 
of  an  individual  command.  These  functions  are 
typically  tools  which  are  used  in  implementing  the 
actual  operational  programs  which  are  part  of  the 
Command  Support  System.  Many  of  them  do  not 
interface  directly  with  using  personnel.  Where  they 
do,  they  can  usually  be  designed  so  that  the  specific 
language  used  in  the  interaction  can  be  (and  should 
be)  tailored  to  the  needs  of  the  specific  using  per¬ 
sonnel.  These  capabilities  provide  the  following 
functions: 

L  Monitoring  the  operation  programs  which 
actually  carry  out  the  desired  tasks  (the  execu¬ 
tive  program). 

2.  Storing  and  retrieving  data.  This  capability  will 
provide  for  determining  what  information  is  to 


be  stored,  from  what  sources  it  arises  for  actually 
storing  the  data,  and  for  retrieving  data.  Two 
elements  are  important  in  describing  the  data 
and  their  sources.  The  first  is  a  capability  for 
adding  or  deleting  elements  of  information  from 
the  files  and  the  second  is  providing  technical 
or  administrative  capabilities  to  delimit  those 
individuals  who  may  change  the  file  structure. 
When  data  are  to  be  inserted  into  the  files,  they 
may  arise  from  processing  of  reports  or  tele¬ 
communications  entering  the  headquarters  or 
as  the  result  of  operations  carried  out  within  the 
headquarters.  In  either  case,  the  system  must 
provide  the  capability  to  accept  data  from  an 
appropriate  source.  It  must  provide  a  capability 
to  change  the  source  or  conditions  under  which 
information  may  be  accepted  and  it  must  limit, 
by  administrative  or  technical  means,  those  in¬ 
dividuals  who  can  so  change  the  system.  The 
system  must  provide  a  powerful  retrieval  capa¬ 
bility  to  retrieve  information  from  these  files. 
To  accomplish  this,  a  capability  to  actually 
carry  out  the  retrieval  is  required.  This  must 
be  supplemented  by  a  control  mechanism  to 
limit  which  information  may  be  retrieved  by 
what  individuals  and  again  there  must  be  a 
mechanism  for  identifying  who  may  change  the 
system. 

3.  Processing  the  information  which  has  been  re¬ 
trieved  from  the  files  in  moderately  complex 
manner.  This  capability  includes  both  the  pro¬ 
cessing  tools  and  a  set  of  delineators  which 
determine  who  may  add  processing  capabilities 
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to  the  system  or  may  modify  the  existing  capa¬ 
bilities. 

4.  Tools  (compilers,  debugging,  editing,  etc.)  for 
producing  new  programs  to  be  added  to  the 
existing  complement,  together  with  the  necessary 
controls  to  prevent  unauthorized  changes  in 
the  system's  program  structure. 

It  seems  to  me  that  within  this  framework  a  com¬ 
mand  system  can  be  implemented  respective  to  the 
user  requirements  and  which  meets  Mr.  McFee's 
constraints  with  respect  to  the  design  of  the  command 
and  control  system.  I  believe  that  the  specifics  inter¬ 
acting  with  programs  which  I  have  just  discussed  will 
typically  be  hidden  from  most  of  the  users  in  the  real 
system,  such  that  the  requirements  to  tailor  the 
language  used  for  performing  these  functions  to  the 
command  is  not  really  necessary,  which  I,  therefore, 
believe  also  conforms  to  Mr.  Wessell's  objections  to 
laboratory-designed  subsystems.  Since  Mr.  Vance 
appears  to  be  more  convinced  than  I  of  the  feasibility 
of  carrying  out  laboratory  designs,  I  feel  that  he  also 
concurs.  The  element  around  which  we  had  planned 


to  center  our  discussion  was,  therefore,  precisely 
who  and  how  and  when  do  the  parameters,  which 
are  left  open  by  the  system  components  I  have  just 
described,  become  established. 

The  other  question  is  the  extent  to  which  having 
built  the  system  in  the  laboratory  facilitates  the  de¬ 
velopment  process  in  the  field.  I  believe  that  it  at 
least  halves  the  time  required  to  get  a  system  operat¬ 
ing  in  response  to  the  requirements  of  the  command. 

The  final  question  which  I  had  planned  to  raise,  if 
there  were  time,  is  a  discussion  as  to  whether  the 
capabilities  that  I  have  just  described  might  not 
facilitate  the  problems  of  intersystem  communication 
and,  particularly,  of  the  fact  that  the  over-all,  world¬ 
wide  command  system  is  not  implemented  as  a  mono¬ 
lith  but  rather  grows  by  evolution  of  its  independent 
parts.  I  submit  that  the  capabilities  required  to  pro¬ 
vide  the  needed  flexibility  within  the  command  also 
go  a  considerable  way  towards  solving  those  same 
problems  for  the  inter-command  communications 
problem. 


Laboratory  design  of  command  and  control  systems 


by  Donald  L.  Drukey 

System  Development  Corporation 
Santa  Monica,  California 


INTRODUCTION 

The  question  I  would  like  to  address  is  the  extent  to 
which  it  is  feasible  to  design  a  command  and  control 
system  for  a  military  headquarters  at  the  facilities  of 
the  developing  organization.  My  opinion  is  that,  be¬ 
cause  of  variations  between  military  headquarters  and 
the  close  user  involvement  required  in  the  design  of 
many  system  functions,  it  is  not  possible  to  develop 
an  entire  system  in  the  laboratory.  However — and 
this  is  the  major  point  of  my  thesis — there  arc  certain 
common  elements  of  such  systems  which  arc  known 
in  advance  and  which  will  provide  a  set  of  general- 
purpose  tools  that  can  much  shorten  the  process  of 
implementing  a  command  and  control  system  at  the 
user’s  facility. 

For  this  discussion,  the  term  “command  and  con¬ 
trol”  should  be  qualified  to  that  of  “command.”  By 
“command”  I  mean  those  functions  of  a  headquarters 
that  arc  concerned  with  planning  and  carrying  out 
military  operations  a  day  or  more  in  advance,  moni¬ 
toring  the  operations  as  they  unfold,  and  making 
changes  in  the  plan  to  meet  the  exigencies  of  the 
situation.  I  specifically  exclude  the  control,  or  “real 
time,”  function  which  is  addressed  to  producing  direct 
controlling  instructions  to  weapons  and  defensive  sys¬ 
tems.  A  great  deal  of  the  confusion  with  respect  to 
command  and  control  may  be  due  to  an  unwillingness 
to  admit  that  these  tend  to  be  separate  functions.  The 
control  problem  is,  of  course,  far  from  trivial.  How¬ 
ever,  for  present  purposes,  I  am  addressing  the  prob¬ 
lems  of  a  headquarters  which  either  has  a  small 
contingent  responsible  for  such  real-time  control  or 
which  relegates  this  area  to  subordinate  headquarters. 

The  problem 

The  functions  that  I  want  to  discuss  arc  those  of 
providing  support  to  the  commander  and  his  staff  in 
their  day-to-day  and  longer-term  planning.  The  reac¬ 
tion  time  for  these  operations  is  typically  measured 
in  hours  or  even  longer.  It  is  primarily  to  personnel 
operating  on  such  a  time  cycle  that  command  systems 


automation  can  provide  real  support.  These  personnel 
arc  concerned  with  data  problems — in  particular,  with 
problems  that  arise  when  one  has  large  volumes  of 
information  that  are  or  may  be  pertinent  to  the  deci¬ 
sions  to  be  made.  Often  the  information  required  to 
assist  in  making  these  decisions  is  in  the  headquarters 
but  is  unavailable,  because  it  is  inaccessible  or  because 
it  cannot  readily  be  put  together  in  the  form  that  is 
responsive  to  the  particular  problem. 

An  automated  command  system,  then,  should  pro¬ 
vide  the  capability  for  receiving  inputs  to  such  a  file 
of  information,  for  storing  it  away,  for  updating  and 
validating  it  as  appropriate,  for  retrieving  information 
from  the  file,  and  for  performing  the  required  opera¬ 
tions  on  that  information  to  make  it  addressive  to  the 
problem  at  hand.  This  last  requirement  is  crucial 
to  an  adequate  solution  to  the  command  problem. 
Simply  providing  large  files  of  information,  even  with 
a  relatively  prompt  but  inflexible  retrieval  means,  can 
result  only  in  flooding  the  decision  makers  with  large 
amounts  of  largely  irrelevant  data.  Wc  need  the  capa¬ 
bility  to  retrieve  flexibly  and  to  process  the  data  into 
a  form  easily  assimilated  by  the  using  personnel. 

Characteristics  of  command  headquarters 

Let  us  inquire  whether  there  are  underlying  attri¬ 
butes  that  arc  common  to  many  headquarters  and  that 
arc  invariant  as  the  personnel  within  a  headquarters 
change  or  as  the  mission  of  the  headquarters  changes. 
I  assert  that  there  arc  such  constant  factors  and  that 
by  addressing  ourselves  to  them  wc  can  produce  a 
system  which,  prior  to  installation  in  the  headquarters, 
has  solved  many  of  the  problems  that  have  proven 
very  time  consuming  and  difficult  in  the  past  when 
done  in  the  operational  context. 

As  we  compare  various  headquarters,  certain  dif¬ 
ferences  are  immediately  apparent.  Almost  no  two 
headquarters  have  files  with  the  same  structure,  al¬ 
though  some  elements  of  the  files  may  be  widely 
shared  between  headquarters.  The  amounts  of  infor¬ 
mation  stored  and  received  in  a  headquarters  are 
highly  variable.  The  processing  of  data  to  make  them 
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meaningful  to  the  using  personnel  is  highly  dependent 
on  the  mission  of  the  headquarters  and  on  the  person¬ 
alities  and  capabilities  of  the  headquarters  personnel. 

On  the  other  hand,  eertain  eharaeteristies  are  con¬ 
stant  throughout  the  community: 

•  large  quantities  of  information  must  be  filed; 

•  the  precise  nature  of  the  retrievals  can  only 
partially  be  specified  in  advanee; 

•  over  time,  different  elements  assume  different 
importance  in  the  files,  and 

•  data  selection  and  formatting  must  be  respon¬ 
sive  to  ehanges  in  personnel  and  in  mission. 

Considering  the  problem  historically,  and  foeusing 
on  the  large  L-systems,  we  find  that  they  consist  of 
three  elements  in  addition  to  the  personnel:  a  com¬ 
plement  of  hardware  for  the  communications,  com¬ 
puting,  and  display  functions;  an  aggregate  of  com¬ 
puter  programs  to  control  the  operations  of  that  hard¬ 
ware;  and  the  procedures  used  by  operational  person¬ 
nel  to  direct  the  hardware  and  computer  programs.  It 
is  largely  through  these  procedures  that  the  user  ean 
exercise  whatever  flexibility  remains  within  the  hard¬ 
ware  and  software  environment. 

More  often  than  not,  the  press  of  system  develop¬ 
ment  schedules  has  left  the  developers  with  no  ehoiee 
but  to  provide  a  system  that  leaves  little  flexibility  for 
the  users  or,  if  it  is  there,  provides  it  in  an  extremely 
awkward  way.  We  find  that  the  hardware,  particularly 
the  computing  elements,  contains  an  inherently  large 
but  finite  degree  of  flexibility,  which  is  constrained  by 
the  software.  When  a  headquarters  needs  to  adapt  to 
changing  situations,  it  attempts  to  modify  its  procedures 
and  to  utilize  what  little  flexibility  remain  in  the  hard¬ 
ware/software  system.  The  personnel  are  limited  in 
their  ability  to  exploit  this  flexibility  by  the  extent  to 
which  they  know  and  understand  the  operations  of  the 
hardware  and  software  system.  Typically,  they  do  not 
understand  them  well  enough  to  make  on-the-spot  ad¬ 
justments  effectively,  so  they  end  up  by  making  entirely 
manual  fixes  and  bypassing  those  parts  of  the  system 
which  they  cannot  readily  modify  to  suit  their  own 
needs. 

Generalized  software  support 

Of  the  complement  of  computer  programs  that  op¬ 
erates  in  a  command  headquarters  environment,  a  very 
large  portion,  sometimes  well  over  half,  is  concerned 
with  tools  for  producing  and  eheeking  out  computer 
programs — in  short,  for  the  means  of  modifying  the 
software  part  of  the  system.  The  remainder  consists 
of  operational  programs  that  provide  for  retrieval  of 
data  and  for  processing  these  data  in  response  to  the 
specific  requirements.  The  general  paekage  that  I 
would  propose  be  developed  in  advanee  of  implemen¬ 


tation  of  the  system  at  a  headquarters  will  take  eare 
of  the  support  functions  and  will  provide  a  number 
of  the  tools  whieh  make  the  retrieval  and  processing 
of  data  easy.  On  the  other  hand,  I  do  not  believe  that 
much  ean  be  done  in  the  laboratory  to  actually  tailor 
these  tools  to  the  operational  needs  of  the  headquarters. 

These  operational  programs  have  to  be  jointly 
worked  out  with  the  actual  using  personnel  for  two  rea¬ 
sons:  first,  user  personnel  find  it  difficult  to  formalize 
their  requirements  for  a  capability  whieh  they  may 
as  yet  only  dimly  understand,  and  second,  many  of 
the  requirements,  particularly  for  output  formats,  are 
rather  personal  and  are  not  an  invariant  to  be  settled 
onee  and  for  all. 

What,  then,  ean  be  done  before  a  system  is  installed 
and  completed  at  the  user’s  site?  I  believe  that  flexible 
hardware  ean  be  provided  with  a  memory  large  enough 
for  an  initial  installation  and  capable  of  expansion 
as  the  requirements  increase  with  time  (whieh  they 
invariably  do).  Then,  I  believe  that  we  ean  provide 
basie  software  which  the  user  will  treat  as  though  it 
were  a  part  of  the  hardware,  that  is,  something  whieh 
is  maintained  by  the  supplier  and  whieh  the  user  does 
not  need  to  modify.  This  basie  software  holds  the  an¬ 
swer  to  the  flexibility  problem — by  providing  the  capa¬ 
bility  for  the  user  to  easily  implement  the  operational 
portions  of  his  system  and  to  modify  them  with  little 
or  no  support  from  the  professional  programming 
community. 

The  ingredients  that  belong  in  such  a  software  com¬ 
plement  inelude: 

•  A  time-shared  executive  whieh  permits  the  user 
to  earry  on  program  development  and  experimen¬ 
tation  concurrent  with  his  operations,  and  to 
link  a  number  of  remote  data  sources  to  his 
computer. 

•  A  general-purpose  data  base  handling  system 
whieh  makes  it  easy  for  his  operational  personnel 
to  specify  whieh  data  to  inelude  in  their  data 
base,  the  form  in  whieh  they  wish  to  insert  those 
data,  and  the  modification  to  these  descriptions 
as  eireumstanees  indicate.  This  data  base  handling 
system  would  also  permit  easy,  flexible,  user- 
oriented  retrieval  of  information,  and  would  pro¬ 
vide  for  convenient  loading  and  updating  of  the 
data  base. 

•  A  reporting  and  display  system  which  makes  it 
easy  for  operational  personnel  to  specify  formats 
for  reporting  and  for  display  of  information  and 
provides  a  simple  complement  of  easily  used 
arithmetic  capabilities  for  processing  data  re¬ 
trieved  from  the  data  base  prior  to  display  or 
report  preparation. 

•  A  higher  order  language  capability  for  producing 
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efficient  operating  programs  specific  to  user 
needs. 

•  A  set  of  conveniently  usable  tools  for  debugging 
and  cheeking  out  programs  both  from  the  stand¬ 
point  of  the  nonprogrammer,  data-oriented  user 
and  from  the  standpoint  of  the  professional  pro¬ 
grammer. 

All  these  capabilities  are  within  today’s  technology. 
We  have  systems  in  existence  today  in  a  variety  of 
locations  which  provide  each  of  these  capabilities.  Some 
of  them  are  further  developed  than  others.  Program¬ 
ming  languages  and  executive  systems  are  in  pretty 
good  shape.  The  general-purpose  data  base  handling 
systems  and  report  generating  and  display  systems, 
while  they  exist,  tend  to  be  poorly  human-engineered 
in  their  present  forms  for  an  operational  environment. 
To  really  satisfy  the  requirements,  they  should  be  sim¬ 
ple  and  straightforward  enough  so  that  a  reasonably 
intelligent  user,  familiar  with  the  data  that  he  wishes 
to  process,  can  learn  to  operate  the  system  in,  at  most, 
a  week  or  two  of  part-time  instruction.  We  seem  to 
be  on  the  verge  of  being  able  to  provide  that  capability. 

The  primary  limitation  to  our  progress  with  such 
systems  is  not  lack  of  technical  facilities  but  the  faet 
that  the  systems  are  not  yet  convenient  enough  to 
attract  sizable  numbers  of  operational  users.  As  soon 
as  we  get  more  such  users  we  can  shake  down  the 
systems  faster  and  turn  them  into  really  useful  tools. 
Then  we  can  begin  implementation  of  a  general-pur¬ 
pose  software  package  having  these  basic  capabilities. 
Such  a  package  would  shorten  the  development  cycle 
for  tailoring  the  command  and  control  system  to  the 
actual  requirements  of  a  headquarters  by  a  factor  of  2 
or  more,  and  the  impact  on  the  ability  of  that  head¬ 
quarters  to  evolve  its  own  system  would  be  even  more 
substantial. 

Installation  of  the  generalized  software 

I  would  like  to  discuss  the  mechanism  by  which 
such  a  capability  could  be  phased  into  the  operations 
of  the  headquarters.  Let  me  postulate  that  the  hypo¬ 
thetical  headquarters  is  now  operating  with  a  system 
utilizing  an  IBM  1410  and  uses  that  headquarters’ 
variant  of  the  473L  System.  The  system  consists  of  a 
set  of  files,  a  set  of  the  basic  procedures  supplied 
through  473L  essentially  unmodified,  a  second  set  of 
procedures  that  have  been  adapted  with  minor  modi¬ 
fications  for  the  uses  of  the  headquarters,  and,  finally, 
a  significant  collection  of  programs  that  is  specific  to 
the  headquarters  and  was  developed  there  to  meet  its 
own  needs. 

The  basic  software  package  that  I  have  described 
previously  will  provide  the  equivalent  capabilities  of 
the  unmodified  programs  from  473L.  Typically,  these 


are  the  programs  associated  with  inputting,  storing,  and 
updating  information,  and  retrieving  from  the  data 
base.  The  phasing  in  of  the  new  system  will  consist 
initially  of  replicating  most  of  the  externals  of  the 
capabilities  available  to  the  staff  personnel  within  the 
framework  of  their  old  system.  After  the  staff  is  con¬ 
vinced  that  the  new  system,  operating  in  parallel  with 
their  old  system,  provides  the  same  or  comparable 
capabilities  and  is  easy  and  straightforward  to  use,  the 
old  system  will  be  phased  out  and  the  process  of  modi¬ 
fying  the  new  system  to  exploit  its  enhanced  capabili¬ 
ties  will  begin. 

Replicating  the  old  system  starts  with  the  ability  to 
store  the  information  contained  in  the  files  of  the  old 
system  and  to  retrieve  from  that  store.  It  should  be  a 
straightforward  process  to  generate  file  descriptions  for 
the  categories  of  information  contained  within  the  old 
file  by  a  system  such  as  the  data  base  system  described 
above.  Taking  the  actual  file  of  information  from  the 
1410  System  and  converting  it  to  the  new  data  base 
system  is  somewhat  more  complicated.  Although  this 
requires  the  services  of  a  professional  programmer  who 
understands  the  structure  of  the  old  file,  it  is  not  a 
terribly  complex  task  and  is  one  that  has  been  done 
frequently.  Since  the  new  system  makes  it  easy  to 
change  the  structure  of  the  files,  it  seems  desirable 
to  initiate  the  changeover  by  preserving  the  old  file 
structure  as  seen  by  the  data  user  (not  the  program¬ 
mer)  within  the  new  system,  and  adding  capabilities 
and  evolving  them  only  after  the  new  system  had 
demonstrated  its  ability  with  the  old  structure. 

Next,  the  process  of  trying  to  replicate  the  capabil¬ 
ities  of  the  operational  programs  for  the  1410  System 
within  the  new  system  can  begin,  starting  with  the  less 
complex  ones.  Much  of  the  complexity  of  the  old  pro¬ 
grams  relates  to  the  now  straightforward  tasks  of  infor¬ 
mation  retrieval  and  display.  These  functions  can 
usually  be  readily  accomplished  with  the  general-pur¬ 
pose,  user-oriented  tools  provided  in  the  new  system. 
This  process  should  be  undertaken  jointly  by  the  imple¬ 
menting  organization  and  personnel  familiar  with  the 
old  system  so  that  reasonable  tradeoffs  can  be  made 
between  slavish  copying  of  the  old  query  capabilities 
and  different  but  simpler  capabilities  available  through 
the  new  system.  As  the  capabilities  come  into  being 
they  will  be  tried  in  parallel  with  the  old  capabilities 
and  it  can  be  demonstrated  that  they  do  (or  do  not) 
produce  the  same  result.  Personnel  will  become  famil¬ 
iar  with  the  new  and  slightly  different  ways  of  doing 
business  and  may  even,  at  this  early  stage,  begin  to 
experiment  with  modified  ways  of  performing  some 
of  their  tasks. 

Finally,  the  process  of  converting  the  remaining,  and 
typically  command-specific,  programs  to  the  new  sys- 
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tern  begins.  Personnel  from  outside  the  headquarters 
will  probably  be  needed  to  supplement  the  headquar¬ 
ters  staff  for  this  task.  This  conversion  should  begin 
slowly,  since  a  great  deal  of  programming  will  probably 
be  required  and  it  would  be  a  shame  to  tie  this  too 
firmly  to  the  old  mode  if  the  new  capabilities  offer 
greater  power.  1  would  recommend  that  the  opera¬ 
tional  programs  be  priority-ordered  and  the  most  im¬ 
portant  ones  taekled  more  or  less  one  at  a  time  so 
that  the  learning  process  can  be  brought  to  bear  on 
later  portions  of  the  conversion  process  as  early  as 
possible.  Bceause  the  time-shared  executive  enables 
several  users  to  have  access  to  the  system  at  one  time, 
there  need  be  no  interference  between  development 
and  implementation. 

CONCLUSION 

The  proeess  I  have  just  described  holds  many  potential 
problems  with  respect  to  reliability.  It  is  very  impor¬ 


tant  that  the  general-purpose  tools  provided  by  the 
new  system  be  extensively  cheeked  out  before  the  con¬ 
version  process  is  attempted.  Inadequacies  of  the  tools 
necessarily  reduce  user  acceptance  of  the  new  system. 
This  is  particularly  true  of  the  executive  system  which 
can  clobber  the  parts  of  the  operational  program  that 
the  command  staff  might  like  to  exploit. 

In  conclusion,  I  should  point  out  that  there  is  a  very 
serious  problem,  particularly  in  the  Air  Foree,  in 
bringing  such  a  package  into  being.  In  principle,  it 
is  ESD’s  charter  to  produee  sueh  tools  but,  for  what¬ 
ever  reason,  it  has  proven  virtually  impossible  for 
ESD  to  achieve  the  funding  support  required  to  eon- 
tract  for  such  efforts.  Until  one,  or  preferably  more 
than  one,  effort  to  provide  sueh  capability  oceurs,  this 
will  be  talk  and  not  action  and  we  will  never  be  able 
to  find  out  whether  these  concepts  held  by  many  of 
us  are  valid. 
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“ All  twelve  violins  were  playing  identical  notes . 
This  seems  to  he  unnecessary  duplication .  The 
staff  of  this  section  should  he  drastically  cut.  If 
a  larger  volume  of  sound  is  required ,  it  could  he 
obtained  by  means  of  electronic  apparatus 
(Author  unknown,  1955) 

INTRODUCTION 

I  quote  part  of  this  study  because  it  is  an  excellent  ex¬ 
ample  of  what  can  happen  when  classical  analysis 
methodology  is  applied  by  an  outside  analyst.  It  was 
contained  in  the  report  of  a  work  study  engineer  after 
his  study  of  a  symphony  concert  at  the  Royal  Festival 
Hall  in  London,  This  report  was  probably  based  on  in¬ 
terviews,  visits,  simulations,  and  other  activities 
which  can  be  done  in  a  laboratory  environment  with 
a  minimum  amount  of  disruption  to  the  user's  en¬ 
vironment. 

I  realize  it  is  an  exaggerated  point;  but  it  is  the  out¬ 
come  which  we  must  continually  guard  against  when 
we  try  to  design  automated  support  for  an  organiza¬ 
tion  without  a  thorough  understanding  of  the  inner 
workings  of  that  organization.  I  use  it  only  to  open  my 
part  of  the  discussion  on  Command  Systems  Simula¬ 
tion  and  Design. 

If  you  look  at  the  agendas  for  the  last  two  Informa¬ 
tion  System  Science  Congresses,  you  will  notice 
that  we  have  had  sessions  similar  to  this;  and  if  you 
read  back  through  the  papers  that  were  presented  and 
remember  the  discussions  at  each  of  these  sessions, 
you  will  find  that  they  cover  a  wide  range  of  subjects. 

The  ends  of  the  spectrum 

The  Congress  discussions  and  papers,  as  well  as 
the  trends  in  system  design  technology  over  the  last 
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six  years,  have  been  controversial.  There  seems  to 
have  been  two  general  philosophies,  and  they  have 
been  quite  divergent  and,  in  fact,  have  ranged  some¬ 
where  between  two  ends  of  a  wide  spectrum. 

At  one  end  of  this  spectrum  has  been  the  philosophy 
that  because  of  many  problems —  administrative, 
technical,  security,  etc.  — it  is  going  to  be  impossible 
to  allow  the  technicians  to  become  intimately  involved 
in  the  day-to-day  operations  of  a  command  or  corpora¬ 
tion.  Therefore,  the  only  alternative  is  to  conduct 
surveys,  interviews,  study  documentation  and  organi¬ 
zations  in  the  abstract  and  to  make  heavy  use  of  simu¬ 
lation  in  a  laboratory  environment.  All  of  these  arc 
used  to  produce  an  ultimate  flexible  system  with 
many  general  purpose  characteristics  which  can  be 
installed  in  a  customer's  environment.  Without  any 
connotation  to  any  current  political  philosophies, 
and  for  purposes  of  this  paper,  let  us  call  this  the 
“left"  of  the  spectrum. 

The  best  example  that  I  could  find  of  this  “left" 
position  was  reported  in  a  trade  journal.  I  have  deleted 
any  reference  to  the  well-known  systems  design  or¬ 
ganization  involved. 

“ Secondly ,  .  .  .  hopes  to  avoid  the  sad  experience 
learned  from  SAGE .  After  SAGE  was  installed, 
the  operator  discovered  that  information  he  often 
needed  was  not  available  to  him .  It  could  not  he 
requested ,  could  not  be  displayed,  nor  w  as  the 
computer  able  to  produce  it.  By  a  series  of  new' 
developments  not  yet  disclosed ,  .  .  .  plans  to  de¬ 
sign  flexibility  into  the  system  and  into  the  pro¬ 
gramming  of  the  system ,  that  will  make  it  auto¬ 
matically  responsive  to  new,  unforeseen  require¬ 
ments  as  they  emerge ,  the  system  will  he  able  to 
change  itself  as  it  operates ."  (Mason,  1963) 

At  the  other  end  of  this  spectrum  (the  “right")  is 
the  philosophy  that  only  through  an  involvement  with 
the  user,  and  with  the  user  himself  participating  to 
the  fullest  with  the  designer  in  his  own  environment, 
can  any  progress  be  made.  The  advocates  of  this  posi- 
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tion  propose  that  the  technician  move  into  the  user’s 
environment.  They  advocate  that  it  takes  a  continuous 
process  in  which  the  operator  and  the  technician  work 
together  learning  each  other’s  problems  as  a  team 
effort,  while  they  develop  and  define  areas  for  support 
and  feasible  ways  of  doing  the  job  from  a  technical 
standpoint. 

1  happen  to  feel  that  this  was  closer  to  what  Dr. 
Fubini  meant  in  his  opening  address  to  this  Congress 
two  years  ago: 

“The  purpose  is  not  simply  defined  by  simple 
statements.  I  cun  saying  to  you  gentlemen  — do 
not  overplan;  do  not  design  the  software  as  if  you 
know  what  the  user  wanted ;  bring  the  user  right 
into  the  middle  of  your  machine  and  make  him 
work  with  it,  and  yon  II  be  surprised  at  how  good 
the  man  is.  .  .  .  This  is  why  the  DOD  has  recent¬ 
ly  changed  the  rules  on  military  systems .  We  try 
not  to  deal  with  Weapons  Systems  alone ,  but  to 
bring  the  users  in.  We  want  to  increase  interac¬ 
tion— continuous  interaction  between  the  user 
and  the  doer  ”  (Fubini,  1 964) 

The  philosophies  at  each  end  of  this  spectrum  have 
their  advantages  and  their  disadvantages.  I  need  not 
point  these  out  to  this  audience  as  most  of  you  have 
been  involved  in  systems  development  somewhere 
along  this  spectrum.  In  fact,  the  very  purpose  of  this 
session  is  to  discuss  the  extent  to  which  success  can 
be  achieved  and  how  far  towards  the  “left”  or  the 
detached,  laboratory  approach,  designers  can  oper¬ 
ate.  Of  course,  to  be  practical,  somewhere  between 
these  two  divergent  views  there  must  be  a  trade-off 
point.  Taking  the  particular  command,  the  particular 
improvement  desired,  and  the  particular  financial, 
personnel,  technical  limitations  which  have  been  im¬ 
posed,  one  can  arrive  at  a  “best  fit”  to  the  situation. 

My  position  in  this  paper  is  to  support  the  “right” 
—  if  not  to  convince  you  completely,  at  least  to  swing 
the  pendulum  as  far  in  this  direction  as  I  can.  I  am 
speaking  from  a  number  of  years  of  experience  work¬ 
ing  with  various  users  in  their  environment,  or  trying 
to  convince  others  that  this  is  the  only  way  to  work. 

I  have  been  attempting  to  bring  about  an  improvement 
in  the  development  of  various  support  systems  to 
assist  users  in  performing  their  jobs. 

I  have  not  always  been  successful,  and  I  cannot 
come  before  you  with  a  list  of  spectacular,  technically 
elegant,  multi-computer  systems  that  have  been  de¬ 
signed  applying  this  technique.  In  fact,  it  is  difficult 
to  measure  accomplishments  in  this  area,  and  some 
have  a  distorted  view  as  to  what  is  an  accomplish¬ 
ment;  but  the  only  valid  measure  of  success  is  an  im¬ 
provement  seen  through  the  user’s  eyes.  Here,  I 
have  succeeded. 


Some  qualifications 

There  are  some  definitions  and  a  few  qualifications 
that  should  be  made  early  in  this  paper.  First,  I  will 
limit  my  comments  to  what  is  known  today  as  com¬ 
mand  system  design.  To  make  sure  that  you  under¬ 
stand  that  my  comments  are  meant  to  apply  to  the 
design  of  systems  to  support  organizations  that  per¬ 
form  command  functions,  I  would  like  to  discuss 
very  briefly  some  of  these  functions. 

Command  functions  involve  broad  analyses  of 
strategic  problems.  This  involves  allocating  re¬ 
sources;  alerting,  committing  and  assessing  the 
capabilities  of  both  your  own  and  enemy  forces. 
These  functions  require  the  gathering  of  large  amounts 
of  information,  aggregating  this  information,  analyz¬ 
ing  and  processing  it  in  many  different  ways  and  dis¬ 
tributing  it  throughout  an  organization.  Command 
functions  deal  with  ambiguous  circumstances.  Com¬ 
mand  functions  involve  questions  like:  What  are  the 
courses  of  action  open  to  my  enemy  or  competition? 
Command  functions  involve  planning. 

A  staff  must  modify,  suggest,  and  define  the  com¬ 
mand  functions  as  new  situations  and  problems  arise. 
When  a  plan  is  selected,  command  functions  include 
issuing  of  orders.  They  include  monitoring  the  execu¬ 
tion  of  these  orders  against  the  plan.  Finally,  intelli¬ 
gence  may  start  the  cycle  over  again,  generating  a  new 
command,  a  new  plan.  In  summary,  command  func¬ 
tions  are  concerned  with  the  total  management  of 
the  resources  of  a  command  or  a  corporation.  Or¬ 
ganizations  that  perform  these  functions  are  the  com¬ 
mand  systems  themselves. 

I  am  addressing  my  comments  concerning  the  Sup¬ 
port  Systems  that  can  be  designed  to  assist  these 
organizations  to  perform  better  these  functions.  In 
fact,  a  much  better  name  would  be  Command  Support 
Systems.  If  we  did  this,  it  would  help  us  realize  that 
we  are  not  designing  a  Command  System  or  replac¬ 
ing,  or  even  augmenting  one  that  is  in  existence.  We 
are  only  providing  support  to  an  already  existing 
system  that  is  already  performing  the  functions  dis¬ 
cussed  above. 

1  can  speak  from  experience  in  the  military  environ¬ 
ment.  I  have  a  feeling  that  the  things  we  are  dis¬ 
cussing  here  today  are  just  as  pertinent  to  the  in¬ 
dustrial  management  environment  at  the  command 
function  level  as  it  is  to  the  military.  Discussions  with 
friends  in  industry  have  substantiated  this;  but  1  will 
let  you  make  your  own  extrapolation  to  the  industrial 
environment. 

Secondly,  1  wish  to  make  clear  that  I  am  not  talking 
about  research  in  the  information  systems  area.  If 
1  were,  1  think  1  would  take  a  completely  opposite 
view.  One  of  the  real  problems  with  operating  in  the 
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"right”  mode  is  the  tendeney  of  this  environment  to 
inhibit  major  and  long-range  innovation.  The  user  has 
a  job  to  do.  He  is  going  to  be  involved  in  his  present 
position,  probably  for  only  a  short  period  of  time.  He 
wants  to  see  some  payoff  that  will  help  him.  He, 
naturally,  is  going  to  take  a  dim  view  of  any  long-range 
or  any  non-mission-oriented  research.  Yet  there  is  a 
very  great  need  for  research  in  the  command  informa¬ 
tions  systems  area.  We  have  mueh  to  learn  about 
model  building,  gaming  and  simulation,  retrieval,  data 
analysis  techniques,  indexing,  document  storage,  dis¬ 
play  methodology —  to  name  only  a  few.  Software 
research  efforts  need  to  continue,  and  we  definitely 
need  realistic  problems  and  realistic  data  to  use  in 
these  research  efforts.  But  the  user's  environment  is 
not  the  setting  for  this  type  of  work. 

I  am  not  too  eoneerned  about  the  status  of  this  type 
of  research  because  work  in  the  areas  that  I  have  just 
described  ean  be  done  quite  well  in  the  detached 
laboratory  environment.  In  fact,  there  is  mueh  re¬ 
search  of  this  nature  being  sponsored  for  application 
on  non-military  problems.  Government,  libraries, 
educational  groups,  commercial  and  industrial  groups, 
all  have  a  need  for  this  research. 

Another  reason  1  am  not  concerned  about  this  re¬ 
search  stems  from  the  fact  that  I  have  not  found  prob¬ 
lems  in  the  military  that  are  so  different  from  problems 
confronting  the  rest  of  the  world  that  they  require 
independent  research  efforts  for  exclusive  military 
applications.  These  basie  problems  are  in  existence 
everywhere  and  realistic  problems  as  well  as  realistic 
data  ean  be  drawn  from  less  sensitive  areas.  In  faet, 
from  a  tcehnieal  standpoint,  even  more  challenging 
problems  are  in  existence  outside  the  military  en¬ 
vironment.  This  is  not  to  say  that  the  military  does  not 
have  a  need  and  an  obligation  to  sponsor  basie  re¬ 
search  in  eommand  information  systems  area  — it 
just  should  not  be  done  in  the  user's  environment. 

Now  probably  not  many  people  will  argue  with  me 
on  this  subject,  so  why  did  we  get  in  the  position  of 
attempting  or  advocating  research  in  the  user's  en¬ 
vironment?  It  seems  to  have  eome  from  a  problem  of 
semantics,  and  this  was  brought  about  by  some  arti¬ 
ficial  definitions  which  we  had  to  develop  in  order  to 
get  around  some  serious  funding  problems.  Wc  have 
not  in  the  past,  noi  do  we  have  today,  any  really 
good  guide  lines  as  to  how  mueh  to  spend  on  develop¬ 
ment,  and  how  mueh  to  spend  on  research,  hardware, 
ete.  This  problem  is  not  unique  to  the  military  eom¬ 
mand  area  and  is  one  whieh  seems  to  plague  research 
managers  everywhere.  But  in  this  area,  because  the 
technology  was  new  and  managers  did  not  have  the 
understanding  of  the  complexities  involved,  we  have 
been  able  to  get  away  with  some  eolossal  misnomers. 


We  have  had  some  tremendous  developmental  ef¬ 
forts  whieh  have  been  ealled  research  for  funding 
purposes;  and  some  tremendous  research  jobs  whieh 
have  been  funded  under  developmental  budgets. 

Unfortunately,  these  misnomers  have  done  no  one 
any  good.  It  has  eaused  apprehension  on  the  part  of 
some  users  who  were  expeeting  an  operational  sys¬ 
tem,  and  I  am  sure  it  has  drawn  away  some  excellent 
research  talent  and  relegated  them  to  menial  develop¬ 
mental  tasks. 

I  cannot  help  but  remember  a  definition  in  this  area 
that  I  overheard  at  the  2nd  Congress  at  the  Home¬ 
stead.  If  you  remember,  implicit  programming  was 
the  number  one  item  for  discussion.  They  said,  "You 
have  heard  of  problem-oriented  languages  and  you 
have  heard  of  maehine-oriented  languages,  and  pro¬ 
cedure -oriented  languages  — well,  implicit  program¬ 
ming  is  a  fund-oriented  language.” 

1  offer  no  solution  to  these  problems  of  funding, 
and  I  hope  some  of  the  other  sessions  either  at  this 
Congress  or  those  that  will  follow  will  try  to  taekle 
this  problem. 

So  I  am  talking  about  the  development  of  an  opera¬ 
tional  system  — not  research.  A  eertain  amount  of 
experimentation  for  immediate  system  improvement 
is  neeessary  in  the  user's  environment  and  is  the  only 
place  where  it  ean  be  done.  But  research  is  better 
done  elsewhere.  I  will  assume  that  the  research  has 
been  successfully  accomplished,  and  this  research  is 
readily  available  to  the  development  effort. 

In  summary  then,  we  are  discussing  development, 
not  research;  eomnuind  support  systems,  not  com¬ 
mand  systems;  eommand  functions,  not  control  or 
taetieal;  and  we  realize  that  we  have  some  serious 
funding  problems. 

Keep  in  mind  that  my  prime  purpose  for  this  paper 
is  to  eonvinee  you  that  developmental  work  ean  only 
be  successfully  performed  within  the  user's  environ¬ 
ment  on  a  day-to-day  basis,  with  his  staff,  and  that  I 
hold  little  hope  for  any  successful  developmental 
work  being  done  in  the  detached  laboratory  which 
will  have  any  real  payoff  or  any  possible  application 
when  later  installed  in  the  user's  environment.  I  plan 
to  use  the  following  method  to  eonvinee  you  of  my 
position.  In  my  experiences  I  have  eome  across  some 
very  basie  characteristics  of  the  organizations  at  the 
eommand  function  level.  Most  of  these  character¬ 
istics  are  probably  quite  obvious  to  you  all  regardless 
of  whether  you  have  been  working  on  the  inside  or 
the  outside.  What  probably  is  not  so  obvious  to  the 
outsider  is  the  importance  that  intimate  knowledge 
of  these  eharaeteristies  plays  to  the  development  of 
any  system  that  would  support  the  organization. 
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The  hypothesis  that  I  will  leave  for  you  is  that  only 
through  the  type  of  relationship  which  l  am  advocat¬ 
ing  is  one  ever  going  to  be  able  to  determine  these 
characteristics  and  the  parameters  necessary  for 
design  of  a  successful  system.  As  I  explore  a  few  of 
these  characteristics,  ask  yourself,  first  of  all,  the 
necessity  of  knowledge  of  these  characteristics  in 
the  design  of  a  system;  and  second,  the  feasibility 
of  determining  these  relative  parameters  on  the  basis 
of  interviews,  laboratory  simulation,  questionnaires, 
and  documentation.  These  are  the  only  things  that 
are  available  to  my  friends  on  the  “left.” 

The  new'  environment 

Probably  the  most  important  characteristic  of  or¬ 
ganizations  that  needs  to  be  examined  in  the  design 
of  a  command  support  system  is  the  environment 
within  which  these  organizations  have  to  operate. 
Although  much  has  been  written  on  the  subject,  1 
feel  that  few  people  on  the  outside  have  a  real  feeling 
of  the  depth  and  size  of  the  effect  both  in  organiza¬ 
tions  and  facilities  that  has  resulted  from  the  change 
in  environment  over  the  last  10  to  20  years. 

1  am  talking  about  the  complexity  of  the  new  mili¬ 
tary  responsibility  caused  by  the  impact  that  modern 
technology  has  had  upon  our  command  environ¬ 
ment.  Modern  complex  weapons  systems  and  new 
technology  of  warfare  have  contributed  greatly  to 
this  change.  The  number  and  complexity  of  our 
weapons  systems,  the  speed  of  transmission  of  in¬ 
formation,  and  the  advances  in  communication  and 
transportation  technology,  have  brought  closer  to¬ 
gether  the  nations  of  the  world.  This  has  caused  an 
abrupt  change  in  our  basic  concept  of  operations. 
Our  command  organizations  have  continually  striven 
to  evolve  and  change  their  ways  of  operating  to  keep 
pace  with  the  changes  in  environment.  The  change 
has  had  such  a  far-reaching  effect  that  only  those  or¬ 
ganizations  have  been  able  to  survive  which  could 
change  their  objectives,  their  methods  and  their  struc¬ 
tures  rapidly  enough  to  keep  pace  with  the  changing 
environment. 

Not  only  has  this  change  come  about  over  the  past 
20  years,  but  the  situation  is  so  dynamic  that  the  en¬ 
vironment  changes  on  a  day-to-day  basis.  Our  national 
objectives  supporting  a  basic  foreign  policy  position 
have  been  known  to  change  on  a  moment’s  notice, 
affecting  many  thousands  of  people  and  producing  a 
rippling  throughout  all  agencies  of  the  government. 
It  may  have  not  only  military  and  foreign  policy  im¬ 
plications,  but  profound  political  effect,  both  in  this 
country  and  in  others.  Entire  countries  in  which  we 
have  had  little  or  no  interest,  have  come  into  the  spot¬ 
light  overnight.  Obscure  Army  Master  Sergeants  in  a 
far  away  country  become  leaders  of  national  import. 


Events  which  in  the  past  have  had  little  or  not  effect 
on  military  or  political  operations,  all  at  once  be¬ 
come  of  great  importance  at  national  level.  In  order 
to  survive  in  an  environment  such  as  this,  people, 
machinery,  communications,  procedures,  organiza¬ 
tional  arrangements,  have  had  to  be  as  dynamic  and 
responsive  as  the  environment  itself.  The  organiza¬ 
tions  that  are  with  us  today  have  adapted  to  this  en¬ 
vironment.  Many  that  haven’t  are  no  longer  with  us 
or  have  been  relegated  to  positions  of  unimportance 
and  we  hope  will  some  day  disappear  from  disuse, 
because  as  long  as  they  remain  out  of  step  they  can 
only  work  against  the  interests  of  the  government. 

What  1  have  just  said  you  have  heard  before.  But 
1  emphasize  again  that  the  effect  that  this  has  on  com¬ 
mand  support  system  development  must  not  be  fully 
appreciated  by  all  of  the  computer  industry  or  I 
wouldn’t  be  seeing  proposals  that  contain  hardware 
with  Cuba  buttons  or  software  that  requires  modifi¬ 
cations  to  change  the  Cuba  button  to  a  Dominican 
Republic  button.  Our  command  organizations  today 
are  changing  so  rapidly  that  even  if  they  do  have  time 
to  make  up  a  simple  organization  chart,  it  is  obsolete 
by  the  time  it  is  off  the  press.  Documentation  of  pro¬ 
cedures  is  a  rarity.  Some  have  stopped  printing  phone 
books,  and  the  travel  office  has  become  one  of  the 
best  places  to  find  out  “who  is  in  charge.” 

The  implications  that  this  has  to  our  systems  de¬ 
sign  is  basically  this:  If  we  want  to  know  anything 
about  the  environment  of  an  organization,  we  had 
better  move  in  and  live  with  it.  They  cannot  afford 
the  luxury  of  stopping  long  enough  to  tell  system  de¬ 
signers  what  is  going  on.  And  if  they  try  to  send  a 
user  representative  to  the  laboratory,  he  can  tell  us 
only  what  it  was  like  when  he  left,  and  then  only  one 
man’s  view  of  the  organization. 

How  they  work 

Organization  is  the  next  basic  characteristic  of 
commands.  One  must  find  out  how  they  work.  In¬ 
formation  on  this  aspect  of  a  command  is  vital  for  a 
system  design.  But  where  do  we  go  to  find  it?  The 
classical  way  of  doing  this  is  to  look  at  organiza¬ 
tional  charts,  charters,  directives,  executive  orders, 
statements  of  procedures,  rules  and  regulations, 
and  any  other  formal  documentation  on  how  an  or¬ 
ganization  works.  Now  even  if  the  changing  environ¬ 
ment  discussed  above  were  not  true  and  organiza¬ 
tions  did  not  change  and  evolve  as  rapidly  as  they  do 
—  and  if  the  above  things  were  in  existence  and  were 
up-to-date,  how  far  can  they  go  to  help  us  in  under¬ 
standing  the  organization  and  the  operation  of  a  com¬ 
mand?  Based  upon  my  experience,  1  can  say  that  they 
are  almost  useless,  or  at  the  most,  they  lay  only  a 
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very  broad  groundwork  for  what  can  turn  out  to  be  a 
tremendous  data  collection  effort  that  is  necessary 
to  find  out  how  an  organization  really  works.  But  the 
things  mentioned  above  are  the  things  that  are  avail¬ 
able  in  the  outside  laboratory  environment,  and  are 
some  of  the  things  that  one  will  collect  from  brief 
interviews  or  questionnaires. 

Probably  the  most  important  single  thing  that  I 
have  learned  is  that  if  we  are  to  provide  a  command 
with  a  support  system  or  support  aids,  we  must  have 
a  fairly  intimate  knowledge  of  the  inner  workings  of 
an  organization.  Many  of  the  intimate  and  close¬ 
working  relationships  that  exist  in  the  command  en¬ 
vironment  become  known  only  after  long  association 
with  the  command.  These  relationships  are  vital  to 
the  exercise  of  command  and  must  be  dealt  with  in 
any  command  system  designed  to  support  that  com¬ 
mander. 

The  formal  documentation  of  organization  and  pro¬ 
cedures  many  times  tells  only  a  small  part  of  the  total 
story.  Many  of  the  procedures,  if  they  are  not  obso¬ 
lete,  are  impractical  and  people  do  not  use  them.  Many 
procedures  just  don’t  work  and  people  have  learned 
this  and  they  have  long  since  fallen  into  disuse.  Some 
procedures  just  take  too  long  and  people  have  found 
that  they  can  get  away  without  using  them. 

Completely  outside  the  formal  organization  is  the 
informal  or  back-door  method  of  operation.  These 
are  the  personal  relationships,  the  temporary  innova¬ 
tions  that  prove  effective,  the  cross-organizational 
communication  channels,  the  by-passes,  both  up  and 
down.  All  these  have  been  developed  to  meet  better 
the  challenge  of  the  changing  environment.  The  larger 
the  organization,  the  more  of  these  back  door  routes 
are  developed.  Some  of  these  are  legal  and  definite 
improvements  to  these  procedures,  and  soon  they 
may  even  be  formalized.  These,  with  a  little  digging, 
are  not  too  hard  to  discover.  Most  users  are  proud 
of  their  innovations  and  when  they  come  to  know 
you  they  will  be  glad  to  share  them  with  you. 

The  others  are  the  ones  that  are  almost  impossible 
to  discover  — the  illegal  operating  procedures.  When 
I  say  “illegal,”  I  mean,  of  course,  in  relation  to  their 
formal  charter,  etc.  Innovation  in  this  area  is  important 
if  an  organization  is  to  survive.  Knowledge  of  these 
procedures  is  as  important  to  the  system  design  as 
the  legal  ones,  but  this  knowledge  does  not  come 
easily.  Before  a  commander  or  his  staff  is  willing  to 
share  any  of  these  procedures  with  an  outside  group, 
he  must  be  convinced  of  their  intentions  — convinced 
that  they  are  on  his  side.  This  confidence  does  not 
come  from  an  occasional  visit  or  an  interview,  but 
can  only  come  from  a  long  term  personal  involve¬ 
ment  with  each  other.  He  handles  these  procedures 


with  utmost  caution  because,  if  they  were  cut  off  or 
in  many  cases  formalized,  he  could  not  do  his  job  and 
his  organization  would  soon  be  out  of  business.  What 
we  are  discussing  is  his  very  future,  his  career,  and 
what  is  more  important,  the  actual  well-being  of  our 
country. 

Some  of  my  colleagues  may  argue  as  to  whether 
knowledge  of  these  factors  is  really  necessary  to  the 
development  of  a  system.  To  this  I  can  only  say  that 
without  the  knowledge  of  the  detailed  workings  of 
an  organization,  it  is  almost  impossible  to  measure 
progress.  Unless  we  are  able  to  determine  how  an  or¬ 
ganization  really  works  and  be  able  to  determine 
where  we  are  in  relation  to  where  we’re  going,  we 
have  no  way  of  knowing  whether  we-re  improving  the 
situation.  All  too  often  I  have  seen  groups  recom¬ 
mend  changes  which  they  thought  were  improve¬ 
ments  based  upon  formal  organizational  arrange¬ 
ments  only  to  find  out,  too  late,  that  these  changes  and 
many  more  had  long  since  been  made.  Without  this 
intimate  knowledge  we  have  no  way  of  knowing  what 
is  realistic,  what  is  relevant,  what  is  practical,  or  how 
large  a  step  we  should  make  in  improvement.  In  fact, 
in  a  practical  sense,  we  need  to  know  who  really  can 
make  the  decisions  as  far  as  continuing  the  system 
design  effort. 

The  decision  process 

Another  organization  characteristic  is  the  basic  de¬ 
cision  process  itself.  I  assure  you  that  I  have  a  com¬ 
pletely  different  view  today  of  how  decisions  are 
made  in  command  level  organizations  than  I  had  some 
years  ago.  This  is  also  something  that  one  cannot  pick 
up  secondhand.  This  is  something  that  one  has  to  sec, 
to  be  involved  in,  to  have  a  real  understanding  of  how 
decisions  are  made  and  more  particularly,  the  use  of 
information  in  this  process.  This  is  the  aspect  of  the 
process  that  is  of  most  concern  to  system  designers. 
The  ultimate  system  pictures  the  decision-maker  sit¬ 
ting  in  front  of  a  screen,  a  console,  or  in  a  board  room, 
listening  and  viewing  the  most  accurate,  up-to-date, 
well-organized,  carefully  filtered  information  — mak¬ 
ing  various  decisions  which  are  then  handed  down  to 
his  staff.  In  reality,  this  is  not  the  case  at  all.  But 
how  this  process  really  works  in  a  particular  organiza¬ 
tion,  what  information  they  use,  when  they  use  it, 
and  what  they  do  with  it,  is  vital  to  the  design  of  any 
information  system.  It  is,  of  course,  the  key  to  how 
the  organization  works. 

Some  have  tried  to  gain  an  insight  into  this  decision 
process  by  studying  records  of  processes  in  various 
crises.  Here  one  has  tried  to  determine  who  made  the 
decision,  what  the  decision  was,  when  the  decision 
was  made,  who  was  involved  in  it,  and  what  informa¬ 
tion  they  used.  This  process,  in  general,  has  met 
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with  almost  complete  failure.  The  reason  for  failure 
is  important.  It  has  failed  because  it  had  been  as¬ 
sumed  that  decisions  were  made  at  one  particular 
time,  based  on  a  collection  of  the  best  information 
available  at  that  time;  while  in  reality,  the  decision 
process  involved  the  whole  organization  and  many 
other  groups.  The  decision,  although  it  may  be  man¬ 
aged  by  a  commander,  involves  his  entire  staff.  This 
staff  is  continually  making  decisions.  This  staff  is 
recognizing,  filtering,  sorting,  editing,  selecting, 
translating,  inputting  and  outputting,  making  deci¬ 
sions  on  what  to  throw  out,  what  to  leave  in,  what  to 
get  more  of,  whom  to  coordinate  it  with,  and  whom  to 
inform.  Decisions  are  being  made  as  to  relevance  and 
importance.  Information  that  has  been  stored  in  li¬ 
braries  and  information  that  has  been  accumulated 
in  the  staffs  own  minds  maybe  many  years  before, 
are  being  called  upon  and  used.  Mistakes  are  being 
made.  Data  are  being  translated  and  summarized. 
Data  are  being  misinterpreted.  Many  specific  human 
processes  are  taking  place.  Many  of  these  are  difficult 
to  understand,  much  less  record  and  study.  The  staff 
is  trying  to  judge,  is  trying  to  relate,  is  trying  to  re¬ 
member,  is  trying  to  predict,  infer,  create,  associate  — 
all  of  these  involve  decisions.  And  when  we  put  these 
all  together  and  final  decisions  are  made,  they  may  be 
based  on  many  other  factors,  prejudices,  agency  pre¬ 
rogatives,  jealousies  and  the  like,  both  relevant  and 
irrelevant.  Much  of  this  information  in  decision-mak¬ 
ing  progress  is  unrecorded.  Much  of  it  is  done  verbal¬ 
ly.  It's  not  only  the  commander  for  whom  we  must 
provide  this  command  support  system  -  it  is  his  total 
organization. 

I  repeat,  study  of  an  organization  and  its  decision¬ 
making  process  as  described  above  is  not  something 
we  do  secondhand.  It  is  something  that  takes  a  long 
time,  close  involvement,  mutual  trust  — even  to  get  a 
partial  picture  of  what  really  goes  on  within  an  or¬ 
ganization. 

But  these  things  arc  the  very  parameters  needed  in 
the  design  of  an  information  system.  Without  them, 
information  requirements  arc  lacking.  Without  them, 
information  flows,  storage  capacity,  loading  limits  and 
other  necessary  and  very  vital  inputs  for  the  design  of 
a  system  are  meaningless. 

The  importance  of  information 

Although  the  concept  of  information  and  its  im¬ 
portance  to  both  an  individual's  and  an  organiza¬ 
tion’s  existence  has  been  known  for  a  long  time,  we 
seem  to  have  shied  away  from  considering  this  as  a 
basic  design  parameter  as  if  it  were  something  almost 
immoral.  Nevertheless,  it  is  very  clear  that  the  posses¬ 
sion  of  information  is  the  very  essence  of  power  in 
command  level  organizations.  Show  me  the  man  who 


has  exclusive  possession  of  information  in  an  organi¬ 
zation,  and  1  will  show  you  the  man  who  is  the  most 
powerful  in  his  organization,  regardless  of  his  po¬ 
sition.  Although  most  people  will  agree  with  me  on  this 
subject,  few  will  admit  it  openly.  Ignore  it  if  you  want, 
but  it  is  a  fundamental  fact  and  a  real  characteristic 
of  organizations  at  this  level  and  must  be  dealt  with 
by  any  system  designer. 

The  effect  that  information  systems  have  on  the 
distribution  of  information,  and  therefore  on  power, 
is  of  course,  obvious.  Any  change  or  disruption  in 
access  to,  possession  of,  or  distribution  of  informa¬ 
tion  will  be  resisted  by  those  who  may  think  they  will 
lose  power  by  this  innovation.  Intimate  knowledge 
of  an  organization,  how  it  works  and  who  possesses 
information,  is  necessary.  The  ability  to  be  the  sole 
possessor  of  a  piece  of  information  and  to  gain  access 
to  the  boss  because  of  this  information  is  so  important 
to  organizations  at  this  level  that  a  system  designer 
would  do  well  to  remember  this  when  he  proposes 
unlimited,  on-line  access  across  organizational  lines 
to  various  lateral  as  well  as  subordinate  units.  No 
commander  in  his  right  mind  will  allow  such  a  system 
to  operate  unless  he  knows  he  has  full  control  over 
who  has  access,  over  what  he  gets,  and  when  he  gets 
it.  Security  is  only  a  minor  problem  compared  to  what 
I  am  discussing. 

These  are  not  things  fhat  are  talked  about  in  the 
open  — they  are  not  things  that  you  will  come  across 
by  a  casual  association  with  an  organization  — they 
are  things  you  must  dig  for.  Unless  the  user  has  been 
so  involved  in  the  development  of  his  own  system 
that  he  can  assure  himself  that  his  information  is 
being  protected,  he  will  not  use  the  system  regardless 
of  how  well  it  works.  Actually,  this  characteristic 
of  an  organization  can  work  to  the  system  designer’s 
advantage.  Automated  support  can  provide  the  user 
with  more,  better,  and  faster  information,  and  pro¬ 
vide  better  controls  and  better  access  to  information 
than  he  can  possibly  hope  for  without  automation. 
You  cannot  just  tell  him  this.  He  has  to  see  it.  He 
has  to  be  convinced  that  he  can  use  it  to  his  advant¬ 
age. 

Flow  of  information  and  exercise  of  authority 

One  must  also  understand  the  differences  between 
flow  of  information  and  the  execution  of  command 
authority.  Unless  a  system  designer  can  determine 
the  differences  between  flow  of  information  for  these 
two  purposes,  he  cannot  properly  design  a  system  to 
support  the  command.  This  distinction  is  not  easy  to 
understand;  and  it  is  not  just  the  system  designer 
that  has  troubles  with  this  one.  The  user  needs  real 
help  here  too.  Professor  Oettinger  describes  this 
problem  very  well: 
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While  there  are  legitimate  reasons  to  guard 
privacy ,  at  least  part  of  this  concern  arises  from 
a  mistaken  confusion  of  information  gathering 
with  the  exercise  of  authority.  Clearly ,  the  open¬ 
ing  of  information  lines  up ,  down  and  across , 
would  legitimize  a  leaping  over  organizational 
boundaries  that ,  while  essential  for  real  accom¬ 
plishment ,  is  done  nowadays  only  at  official  risk 
and  peril.  Organization  lines  reflect  lines  of  au¬ 
thority ,  but  while  knowledge  is  power ,  the  gather¬ 
ing  of  information  is  not  the  exercise  of  authority. 

It  seems ,  therefore ,  perfectly  proper  for  a  man¬ 
ager  to  leap  several  levels  down  in  search  of  an¬ 
swers,  or  for  a  subordinate  to  leap  across  or¬ 
ganizational  lines  and  occasionally  over  his 
boss ’  head,  so  long  as  decisions  and  orders  travel 
by  normal  channels  and  care  is  taken  to  protect 
legitimate  confidences  such  as,  for  example, 
actual  salary  figures/*  (Octtinger,  1964) 

Resistance  to  this  method  of  operation  is  natural, 
but  1  feel  it  comes  more  from  misunderstanding  of 
the  differences  between  the  two  information  flows 
and  that  they  must  and  can  be  separated.  How  to 
do  this  requires  some  real  ingenuity  and  detailed 
knowledge  of  how  the  organization  works. 

This  method  of  operation  is  part  of  the  New  En¬ 
vironment  and  it  is  here  to  stay.  Dr.  Hollomon 
pointed  out  that: 

“ President  Kennedy  . .  .  has  insisted  not  only  on 
the  right,  but  the  necessity  to  talk  to  those  who 
are  informed,  and  not  only  to  those  who ,  by  some 
quirk  of  accident,  occupy  positions  of  authority .” 
(Hollomon,  1964) 

This  was  not  a  peculiarity  just  of  the  Kennedy  Ad¬ 
ministration,  but  has  become  a  method  of  operation 
throughout  the  government.  It  actually  started  with 
the  change  after  World  War  11  when  this  country 
assumed  a  global  concept  of  operations.  There  is  a 
need  for  current  information  in  each  agency  of  the 
government  at  all  levels.  Who  needs  it,  when,  in  what 
form,  etc.,  are  inputs  for  system  design.  As  you  can 
see,  this  is  not  just  an  internal  problem  for  a  par¬ 
ticular  command.  To  gain  an  understanding  of  the 
interagency,  intercommand  communication  problems, 
you  have  to  be  there  when  it  happens. 

Command  re  so  urces 

When  we  discuss  resources  of  a  particular  com¬ 
mand,  we  normally  think  of  financial  resources. 
When  we  are  writing  a  proposal,  we  always  have  in 
the  back  of  our  mind,  regardless  of  the  job  we’ve 
been  asked  to  do,  the  question  of  just  what  is  realistic 
from  a  financial  standpoint,  and  try  to  hit  within  that 


ballpark.  There  are  many  other  resources  of  an  or¬ 
ganization  which  are  normally  overlooked.  Some  of 
these  resources  we  can  discover  quite  easily,  but 
others  can  only  be  discovered  by  operating  in  the 
user's  environment. 

We  need  a  knowledge  of  the  staff  itself,  its  educa¬ 
tion,  its  background,  its  training,  its  abilities  and  its 
limitations,  its  technical  qualifications  and  experi¬ 
ence,  and  many  other  intimate  factors  and  character¬ 
istics.  Although  this  information  is  not  needed  at  the 
level  of  a  particular  individual,  it  is  needed  in  general 
to  help  determine  the  best  way  to  provide  support 
systems  for  an  organization.  Answers  are  needed  to 
questions  such  as:  How  far  can  one  go  in  formatting 
input?  What  procedures  and  methods  are  conve¬ 
nient?  What  is  realistic  within  current  and  projected 
personnel  limitations?  In  my  role  as  a  systems  evalu¬ 
ator,  1  have  seen  systems  that  would  take  Philadel¬ 
phia  lawyers  to  operate  them  when  only  seamen  are 
available.  1  have  seen  systems  that  would  require  full 
colonels  “mark  sensing"  to  make  them  work.  In  or¬ 
ganizations  at  this  level,  wc  don't  retrain  people  for 
a  particular  job.  Unlike  SAGE,  we  can't  build  a 
simulation  lab  and  train  new  console  operators.  We 
are  dealing  with  busy  and  senior  people.  At  this  level 
we  don't  restrict  them  for  hearing  or  visual  deficien¬ 
cies  because  this  has  little  effect  on  their  ability  as 
commanders,  but  these  deficiencies  have  a  lot  to  do 
with  how  they  use  a  system. 

A  characteristic  of  organizations  at  this  level  is  that 
they  are  top-heavy  with  high-level  people.  There 
never  seems  to  be  enough  clerks,  typists,  and  en¬ 
listed  men  to  go  around.  There  are  some  very  im¬ 
portant  reasons  why  this  is  true,  and  some  equally 
important  reasons  why  this  is  not  likely  to  change. 

1  think  this  comes  from  the  fact  that  there  is  not  nearly 
as  much  routine,  delegatable  work  going  on  at  this 
level  as  one  might  think.  Systems  designers  very  often 
overlook  this  fact. 

Knowledge  of  the  resources  of  a  command  arc 
necessary  if  we  arc  to  determine  what  are  some  of 
the  trade-offs  between  operational  convenience  and 
utility.  It  is  necessary  if  we  are  to  do  any  cost  benefit 
analysis  and  develop  realistic  alternatives.  1  have  not 
yet  found  a  way  by  which  1  can  determine  these  char¬ 
acteristics  of  an  organization  from  the  outside.  The 
problem  of  self-criticism  is  very  important  here. 
Command  level  organizations  are  very  sensitive  to 
criticism.  They  keep  their  mistakes  to  themselves. 
Self-analysis  is  almost  unknown  except  in  a  very  small 
segment  of  an  organization.  This  is  not  only  true  due 
to  the  psychological  and  personal  aspect  of  this  analy¬ 
sis,  but  also  due  to  the  political  and  security  aspect. 
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Security 

Now  let’s  look  at  the  last  characteristic  of  organi¬ 
zations  at  this  level  that  we  will  discuss  — security. 
This  is  something  that  has  plagued  system  designers 
and  all  organizations  that  have  attempted  to  use  sup¬ 
port  from  outside  groups  in  designing  and  building 
systems.  I  realize  that  it  has  been  used  as  an  excuse 
many  times  to  inhibit  innovation.  But  in  spite  of  the 
abuses  of  security,  it  is  real  indeed.  I  have  never  been 
convinced  that  it  is  a  problem  peculiar  to  military  or 
intelligence  groups,  because  the  problem  is  broader 
than  one  of  national  security,  in  its  strictest  sense. 

And  if  we  look  at  it  in  this  broader  context,  I  think 
we  will  see  why  it  is  such  an  important  problem.  We 
will  also  see  why  it  is  one  of  the  characteristics  of 
organizations  at  this  level  which  makes  it  almost  im¬ 
possible  to  use  groups  outside  of  the  administrative 
and  security  control  of  the  particular  command.  To 
get  around  some  of  the  security  problems,  we  have 
tried  to  design  some  systems  using  simulated  data, 
sanitized  scenarios,  and  hypothetical  situations. 
We,  of  course,  have  run  into  the  pitfalls  of  the  tech¬ 
nician  working  on  the  outside  not  knowing  what  is 
reality  and  therefore  not  knowing  if  his  ersatz  data 
is  realistic.  When  we  tried  to  have  the  user  provide 
us  with  realistic  data,  he  did  not  know  enough  about 
the  technical  characteristics  of  the  system  to  be  able 
to  provide  technically  complete  data.  We  have  had  to 
build  and  design  based  on  what  we  thought  it  was  like 
on  the  inside.  I  see  no  acceptable  technical  solution 
to  this  problem.  I  have  yet  to  find  a  system  that  was 
fully  checked  out  on  simulated  data  that  didn’t  hang 
up  the  first  time  I  put  in  a  real  message. 

While  working  within  the  user's  organization,  many 
of  these  problems  can  be  solved.  There  are  areas 
where  security  and  administrative  problems  prohibit 
the  user  from  delegating  his  responsibility  to  an  out¬ 
side  group,  or  even  from  presenting  information  out¬ 
side  his  immediate  area.  In  the  development  of  our 
tactical  systems,  we  were  able  to  get  a  pretty  complete 
knowledge  of  a  small  portion  of  the  total  system. 
But  system  designers  at  the  command  level  become 
intimately  involved  with  the  totality  of  information 
which  is  known  to  only  a  few,  even  inside  the  organi¬ 
zation.  This  information  could  never  be  released  to 
a  group  of  outside  technicians.  A  commander  does 
not  release  this  information  to  just  anyone,  regardless 
of  his  security  clearances.  But  by  becoming  involved 


with  his  organization,  working  on  a  day-to-day  basis 
trying  to  solve  the  technical  problems,  a  mutual  trust 
is  developed  and  certain  administrative  controls  can 
be  established,  allowing  the  technician  to  have  access 
to  information  that  could  not  be  generally  released. 

The  technician  must,  in  turn,  give  up  some  of  his 
freedom  in  return  for  possession  of  this  information. 
He  finds  himself  in  a  position  where  it  is  difficult  to 
speak  or  publish  reports  about  his  work.  He  can  easily 
find  himself  restricted  in  his  technical  approach  and 
required  to  attack  the  “brush  fire”  problems  part 
of  the  time.  But  the  access  to  this  information  and  the 
reward  that  a  technical  person  gets  from  the  chance 
to  use  this  information  in  the  application  of  advanced 
technology,  and  the  personal  satisfaction  that  he  gets 
from  seeing  his  development  being  applied  to  real 
problems,  providing  real  solutions,  far  outweigh  any 
of  these  restrictions. 

Not  everyone  will  work  in  this  environment.  But 
1  am  convinced  enough  of  its  importance  to  try  to  con¬ 
vince  others  that  this  is  the  only  approach  to  making  a 
useful  contribution  to  the  problem.  The  “other  way” 
is  not  very  rewarding  or  productive. 

CONCLUSION 

In  summary,  I  have  tried  to  explore  some  of  the  char¬ 
acteristics  of  command  level  organizations  that 
make  operating  in  the  “left”  mode  very  difficult. 
There  are,  of  course,  many  others,  and  I  have  only 
briefly  touched  on  the  problems  that  the  “right” 
mode  of  operation  can  have.  This  is  the  subject  of  a 
whole  paper  itself.  In  all  fairness,  1  must  close  on  the 
point  that  system  designers  are  not  the  only  people 
that  need  convincing  on  this  method  of  development. 
Many  of  the  users  and  a  lot  more  of  the  financial 
managers  still  do  not  support  this  position. 
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INTRODUCTION 

Don  Drukey  has  asked  me  to  discuss  the  “extent 
to  which  a  command  and  eontrol  system  ean  be  de¬ 
signed  on  the  basis  of  interview  and  laboratory 
data  and  other  facts  whieh  can  be  gathered  without 
actually  implementing  the  system  in  the  customer’s 
environment.”  I  should  make  it  elear  at  the  outset 
that  I  am  addressing  the  problem  of  command  system 
development,  and  that  many  of  my  comments  would 
not  apply  to  the  subject  of  control  systems. 

We  might  ask:  Why  after  six  to  eight  years’  ex¬ 
perience  in  the  development  of  command  systems 
are  we  still  addressing  this  sort  of  topic?  The  answer 
is  that  we  have  been  plagued  by  serious  technical  and 
management  problems  in  every  command  system  that 
we  have  attempted  to  implement,  and  have  yet  to 
find  a  really  satisfactory  approach  to  system  de¬ 
velopment.  However,  a  promising  new  approach  is 
evolving  within  the  Air  Force,  based  upon  the  con¬ 
cept  of  a  standard  command  system. 

Most  of  the  impetus  for  the  generation  of  this 
concept  has  eome  from  the  activity  associated 
with  the  installation  of  a  1410  system  at  five  of 
the  major  air  commands.  This  system  was  an  out¬ 
growth  of  the  early  stages  of  473L  and  is  referred 
to  as  the  Air  Force  Interim  Command  and  Control 
System  (AFICCS).  The  AF1CCS  approach  has  been 
to  standardize  on  the  hardware  configuration  and  the 
support  software  — whieh  includes  utility  routines, 
information  retrieval  and  file  building  routines, 
an  executive  program,  etc.  Changes  to  these  items 
are  rigidly  controlled  by  Air  Force  Headquarters. 
However,  the  individual  users  are  free  to  develop 
specific  operational  programs  within  the  constraints 
of  the  standard  hardware  and  support  software  pack¬ 
age.  The  concept  is  simple:  Standardize  on  the  cap¬ 
abilities  that  are  required  by  all  users  such  as  the 
storage  and  retrieval  of  information,  and  the  tools 
for  building  the  speeial  computer  programs  to  meet 
eommand-unique  requirements.  This  eoneept,  1  am 
sure,  will  carry  over  to  the  follow-on  standard  sys¬ 
tem;  and  the  capabilities  will  be  extended  to  include 


such  features  as  a  time-shared  executive  that  will 
permit  multiple  users  at  remote  stations  to  aeeess 
the  data  processor  simultaneously,  an  improved 
programming  language,  better  program  debugging 
aids,  and  the  like.  We  can  also  expect  heavy  emphasis 
on  the  capability  to  upgrade  the  data  processing 
installation  and  still  retain  the  existing  programs, 
without  resorting  to  reeoding  or  to  awkward  emulation 
techniques. 

There  are  two  distinet  aspects  to  the  standard 
system  approach.  One  is  the  technical  development 
of  the  standard  system  itself.  The  other  is  its  appli¬ 
cation  to  the  unique  needs  of  each  eommand.  My 
contention  is  that  a  different  approach  is  required 
for  eaeh  aspeet  of  the  problem.  I  would  like  to  discuss 
these  problems  in  terms  of  requirements  definition, 
system  development,  and  laboratory  support  My 
opinions  are  based  partly  upon  a  recently  completed 
survey  of  our  experience  in  the  development  of  the 
last  generation  of  ”L-Systems,”  and  partly  upon 
direct  involvement  with  several  of  them. 

Requirements 

In  the  past  the  initial  eommand  system  requirements 
statements  were  based  upon  ad  hoc  studies  charac¬ 
terized  by  discontinuity  of  effort  and  superficial 
analysis.  The  analysis  efforts  were  often  directed 
more  toward  supporting  the  development  of  the 
official  requirements  documents  —  QOR,  SOR, 
PSPP,  etc.  —  than  toward  providing  a  meaningful 
input  to  the  system  designer.  These  statements 
typically  contained  broad,  and  sometimes  vague 
definitions  of  requirements.  On  this  basis  specific 
equipment  configurations  were  tailored  for  eaeh 
command,  but  the  details  of  application  to  functional 
problems  were  often  left  as  an  exercise  for  the  soft¬ 
ware  contractor  and  the  user.  Six-  to  seven-year  lead 
times  were  not  uncommon  for  the  implementation  of 
the  systems;  and  the  rather  hazy  functional  require¬ 
ments  did  not  provide  a  good  basis  for  a  detailed 
projection  of  the  system  configuration  and  the  re¬ 
quired  component  eharaeteristies. 
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The  standard  system  approach  attempts  to  avoid 
the  problem  of  tailoring  the  basic  system  components 
to  each  command’s  unique  functional  requirements 
by  orienting  the  design  toward  the  basic  information 
processing  requirements  of  many  users,  and  by  pro¬ 
viding  the  capability  to  expand  the  system  as  the 
user’s  requirements  develop.  My  opinion  is  that 
the  initial  design  will  be  based  upon  data  collected 
on  the  basis  of  interviews  with  the  users,  and  upon 
a  technical  assessment  of  the  state-of-the-art. 

A  method  for  identifying  specific  functional 
applications  of  the  standard  system  is  also  needed. 
My  feeling  is  that  an  on-going  analysis  and  require- 
ments-definition  activity  should  be  established  within 
each  using  command.  For  lack  of  a  better  name,  I  will 
call  this  an  operational  system  improvement  program. 
Such  a  program  would  be  controlled  by  the  user,  and 
directed  toward  identifying  the  operational  improve¬ 
ments  needed  to  enhance  mission  performance,  re¬ 
gardless  of  whether  they  be  implemented  through 
simple  improvements  to  manual  operations,  pro¬ 
cedural  changes,  or  the  application  of  data  auto¬ 
mation  techniques.  The  primary  objectives  of  this  ac¬ 
tivity  would  be  to  state  requirements  in  a  meaningful 
level  of  detail,  and  to  establish  overall  priorities 
for  implementation.  This  activity  would  provide  a 
growing  definition  of  the  information  processing 
requirements  of  the  user.  It  would  be  conducted 
in  the  user’s  environment. 

System  development 

Our  experience  has  shown  that  many  of  the  prob¬ 
lems  encountered  in  the  acquisition  of  command 
systems  were  directly  attributable  to  developmental 
hardware  and  software  items.  In  the  early  conceptual 
and  design  stages,  the  tendency  was  to  overlook  the 
developmental  problems,  to  assume  that  off-the-shelf 
components  were  available,  and  to  schedule  accord¬ 
ingly.  However,  in  spite  of  the  desire  to  make  ex¬ 
clusive  use  of  off-the-shelf  components,  all  of  the 
systems  have  included  developmental  hardware 
items.  In  some  cases,  they  were  required;  and  in 
other  cases,  they  were  inadvertently  chosen  during 
the  source  selection  process.  These  developmental 
items  were  almost  always  late,  or  were  unreliable 
when  first  delivered,  or  both.  The  software  was 
generally  recognized  as  developmental  from  the 
outset,  although  there  was  a  general  tendency  to 
underestimate  the  size  and  complexity  of  the  effort. 
In  addition,  software  production  was  often  plagued 
by  equipment  problems,  and  sometimes  by  inade¬ 
quate  facilities  for  program  checkout.  In  general,  the 
overall  impact  of  component  development  problems 
was  far  less  severe  when  the  acquisition  program 


provided  a  full  system  prototype  before  attempting 
implementation  in  an  operational  environment. 

The  task  of  developing  the  standard  system  is  pri¬ 
marily  a  technical  one,  requiring  a  strong  system 
engineering  effort,  and  should  be  the  responsibility 
of  the  developer.  My  feeling  is  that  the  design  and 
acquisition  activities  should  be  centered  around  a 
full-blown  system  prototype.  No  hardware  or  software 
item,  whether  off-the-shelf  or  developmental,  would 
be  scheduled  for  use  in  the  field  until  it  had  passed 
thorough  acceptance  tests  in  the  prototype  environ¬ 
ment.  Maintenance  and  revision  of  elements  of  the 
standard  system  would  also  be  centrally  controlled 
by  the  developer,  using  the  system  prototype  as  a  test¬ 
bed.  The  testbed  would  also  be  used  as  a  vehicle  for 
training  military  personnel  in  the  technical  aspects  of 
the  systems  operation. 

In  the  terms  of  Mr.  Drukey’s  question,  the  activity 
I  have  just  described  is  not  appropriate  for  the  user’s 
environment,  nor  for  the  laboratory  It  is  an  engineer¬ 
ing  activity  that  stands  between  the  two.  In  fact,  one 
of  its  purposes  is  to  ensure  that  laboratory  items  are 
brought  to  a  reasonable  state  of  engineering  develop¬ 
ment  before  being  considered  for  use  in  the  field. 

Laboratory  support 

So  far  I  have  not  identified  a  role  for  the  laboratory 
in  the  generation  of  requirements,  or  in  the  develop¬ 
ment  of  the  standard  system.  Is  there  a  role  in  the 
application  of  that  system  to  the  unique  requirements 
of  individual  users?  The  key  to  the  answer  hinges  on 
how  well  the  laboratory  can  simulate  the  environment 
in  which  the  applications  must  ultimately  blend.  My 
feelings  on  this  subject  arc  colored  by  two  efforts 
which  1  followed  closely,  and  which  provided  an  op¬ 
portunity  to  compare  the  development  of  data  pro¬ 
cessing  applications  in  the  laboratory  and  in  the  field. 

The  first  effort  is  the  Experimental  Planning  Facil¬ 
ity  that  was  developed  by  MITRE,  and  sponsored  by 
the  473 E  SPO.  The  main  purpose  of  the  activity  was 
to  gain  some  insight  into  the  problems  of  automating 
the  planning  function.  It  was  also  expected  that  the 
first-hand  experience  gained  in  integrating  the  data 
files,  computer  programs,  and  input/output  devices 
would  be  an  important  by-product,  since  473 L  was 
being  implemented  without  the  benefit  of  a  prototype 
development  program.  We  selected  airlift  planning  as 
the  problem  vehicle,  and  established  an  informal 
working  relationship  with  MATS  to  develop  the  oper¬ 
ational  requirements.  The  experimental  system  was 
implemented  on  the  old  SAGE  XD-1  computer  at 
Hanscom  Field,  and  the  initial  capabilities  were  suc¬ 
cessfully  demonstrated  in  early  1962.  The  system  was 
later  extended  to  include  support  to  the  planning  of 
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tactical  fighter/bomber  deployments  requiring  in¬ 
flight  refueling. 

The  question  is:  How  much  did  the  experimental 
activities  help  473  L?  Actually,  the  effort  impacted 
the  473 L  program  much  sooner  than  we  had  antici¬ 
pated.  The  initial  work  of  developing  the  operational 
requirements  for  the  experimental  facility  influenced 
the  specifications  for  an  interim  data  processing  sys¬ 
tem  for  the  Air  Force  Command  Post  that  was  also 
being  sponsored  by  the  473 L  project.  However,  the 
actual  experimental  work  had  very  little  impact.  The 
experimental  system  suffered  from  a  sterile  environ¬ 
ment,  since  it  could  not  compete  for  the  user’s  atten¬ 
tion  with  the  interim  system  that  was  being  developed 
in  his  own  facilities. 

Although  the  experimental  aspects  of  the  laboratory 
program  had  little  effect  upon  473 L,  it  did  have  some 
important  side  effects.  The  experience  gained  in  imple¬ 
menting  the  experimental  system  led  to  the  general 
purpose  data  management  concept  as  personified  in 
the  ADAM  and  CO  LIN  GO  systems,  provided  a  test¬ 
bed  for  some  early  experimental  work  in  PACCS,  and 
educated  a  small  group  of  technical  people  in  one 
aspect  of  the  military  planning  problem.  Ironically, 
Project  492L  (USSTRICOM)  reaped  more  of  the  ben¬ 
efits  than  did  473L  — the  project  that  funded  the  effort. 
The  COLINGO  system  was  implemented  at 
USSTRICOM,  and  two  of  the  people  that  were 
trained  in  the  process  of  developing  the  experimental 
facility  were  able  to  make  an  important  contribution 
in  the  development  of  an  improved  planning  system 
at  that  command.  This  brings  us  to  the  next  effort 
that  I  would  like  to  discuss. 

The  USSTRICOM  planning  system  that  l  just 
mentioned  was  developed  in  the  user’s  environment. 
It  is  used  to  generate  combined  Air  Force  and  Army 
force  lists  with  a  time-phased  deployment  schedule 
for  contingency  operations;  and  to  test  the  feasibility 
of  the  schedule  on  the  basis  of  various  combinations 
of  airlift  and  sealift  capabilities.  A  significant  factor 
in  the  successful  development  of  the  system  was 
that  the  MITRE  technical  people  worked  as  mem¬ 


bers  of  a  military  team,  with  one  of  the  key  users 
leading  the  effort.  The  MITRE  people  were  first  in¬ 
doctrinated  in  the  USSTRICOM  planning  process 
by  assisting  in  the  generation  of  some  actual  con¬ 
tingency  plans.  They  then  assisted  the  military  plan¬ 
ners  in  developing  an  improved  planning  procedure 
that  would  be  amenable  to  data  processing  support. 
Finally,  they  took  the  lead  in  developing  the  specifica¬ 
tions  for  the  data  processing  support  system.  The 
computer  program  was  produced  and  checked  out 
by  military  programmers,  and  has  made  a  real  con¬ 
tribution  to  the  STRIKE  planning  operation.  How 
could  it  miss?  Every  step  of  the  way  the  user  was 
either  leading  the  development  or  approving  design 
decisions.  The  system  had  built-in  acceptance. 

On  the  basis  of  experience,  I  must  conclude  that 
command-unique  data-processing  applications  arc 
best  developed  with  user  participation,  and  in  his 
environment. 

By  restricting  myself  to  a  discussion  of  the  develop¬ 
ment  and  application  of  the  standard  system,  I  have 
given  a  rather  pessimistic  picture  of  the  role  of  the 
laboratory.  Actually,  I  believe  that  laboratory  experi¬ 
ments  can  be  catalytic  in  developing  more  imagina¬ 
tive  application  of  data-processing  techniques  to 
military  problems.  The  example  of  the  Experimental 
Planning  Facility  illustrates  the  importance  of  labora¬ 
tory  work  in  terms  of  stimulating  and  channeling 
technological  development,  although  it  also  points 
out  some  of  the  difficulties  in  attempting  to  justify 
it  in  terms  of  solving  a  specific  user’s  problem  on  a 
schedule.  It  appears  that  laboratory  efforts  should 
not  be  focused  on  short-term  solutions,  nor  should 
they  be  expected  to  produce  products  that  can  be 
transported  directly  to  the  field.  Perhaps  the  most 
effective  way  to  inject  the  laboratory  experience  and 
technical  developments  into  the  field,  and  to  bring 
the  key  user  problems  and  environmental  character¬ 
istics  to  the  laboratory,  would  be  to  rotate  technical 
people  between  applications  projects  within  the 
laboratory  and  field  assignments  within  the  user’s 
environment. 
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Santa  Monica,  California 

A  few  years  ago  S.  M.  Genensky  and  I  wrote  a  paper 
titled,  Some  Thoughts  on  Developing  Future  Command 
and  Control  Systems  .**  This  paper  briefly  states  the 
position  taken  in  that  earlier  paper  in  order  to  raise 
some  questions  as  to  whether  the  newer  capabilities  for 
‘‘on-line”  interactions  between  users  and  automated  sys¬ 
tems  have  outmoded  our  previous  thinking  on  the  sub¬ 
ject  of  command  system  design. 

In  brief,  the  earlier  paper  argued  for  a  version  of  an 
“on-site”  development  and  design  philosophy  supported 
by  a  military  serviee  center  whieh  would  provide  the 
appropriate  specialists  on  loan  to  the  given  user  com¬ 
mand.  Do  the  newer  on-line  software  packages,  the 
so-ealled  “friendly  systems”  which  are  either  currently 
or  about  to  be  available,  permit  the  design  and  develop¬ 
ment  of  command  systems  to  be  returned  to  the  labora¬ 
tory?  As  Don  Drukey  stated  in  his  letter  of  invitation 
to  participate  on  this  panel,  the  question  is  “the  extent 
to  which  a  command  and  control  system  can  be  designed 
on  the  basis  of  interview  and  laboratory  data  and  other 
facts  whieh  ean  be  gathered  without  actually  implement¬ 
ing  the  system  in  the  customer’s  environment.” 

The  heart  of  the  matter  has  not  changed — can  we 
determine  the  system  requirements  in  sufficiently  real¬ 
istic  detail  to  permit  us  to  produce  a  system  design 
capable  of  being  implemented  in  the  time  period  for 
which  the  obtained  requirements  are  appropriate? 

I  quote  my  previous  views  on  this  point: 

.  .  .  Even  with  significant  user  participation,  it  has  not 
been  possible  for  the  system  developer  ...  to  obtain  a 
detailed  description  of  user's  needs  and  operational  re¬ 
quirements  that  could  be  translated  into  a  coherent  func¬ 
tional  design  and  satisfactorily  guide  the  system  designer 
in  the  long-term  development  of  command  and  control 
systems.  .  .  . 

♦Any  v/ews  expressed  in  this  paper  are  those  of  the  author. 
They  should  not  be  interpreted  as  reflecting  the  views  of  The 
RAND  Corporation  or  the  official  opinion  or  policy  of  any 
of  its  governmental  or  private  research  sponsors.  Papers  are 
reproduced  by  The  RAND  Corporation  as  a  courtesy  to 
members  of  its  staff. 
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Furthermore  .  .  .  any  such  description  [of  the  operational 
requirements ]  is  of  necessity  not  invariant  with  respect  to 
time.  .  .  .  [ Before ]  completion  of  a  major  system  develop¬ 
ment  that  might  satisfy  the  specified  operational  require¬ 
ments,  changes  in  national  policy,  military  strategy, 
weaponry,  roles  and  missions,  and  advances  in  technology 
would  have  outmoded  the  developmental  system. 

Historically,  the  attempt  to  resolve  the  basic  difficulties 
in  obtaining  operational  requirements  for  command  and 
control  systems  led  to  the  creation  of  special  agencies 
within  the  military  services  and.  recently,  within  the  De¬ 
partment  of  Defense.  The  overall  management  responsibil¬ 
ity  for  the  development,  acquistion,  and  delivery  of  com¬ 
mand  and  control  systems  was  located  in  such  agencies. 

In  general ,  the  eventual  system  user  had  representatives  in 
the  appropriate  agency  and  was  thereby  able  to  participate 
in  the  system  design  phase.  In  this  ntaner,  the  system 
user  could  introduce  up-dated  functional  requirements  that 
often  resulted  in  program-design  changes.  However,  com¬ 
promise  was  inevitable.  In  order  that  the  established  de¬ 
livery  schedules  might  be  met,  great  pressure  was  applied 
to  freeze  the  system  design  early  so  that  hardware  pro¬ 
curement  and  development  could  begin.  Soon  after  the 
selection  of  the  contractors  there  was  a  tendency  to  fix 
upon  an  initial  operational  capability,  reserving  changes 
for  the  second  phase  of  complete  operational  capability. 

It  was  quickly  discovered  that  most  aspects  of  the  system, 
both  [the  1  hardware  and  software,  soon  became  cast  in 
concrete.  At  this  point,  introduction  of  even  relatively 
small  program-design  changes,  however  important  these 
changes  might  be,  proved  costly  in  terms  of  both  funding 
and  delivery  time.  As  this  experience  became  the  com¬ 
mon  pattern  in  the  broad  systems  approach  to  command 
and  control  developments,  the  need  for  a  change  in 
developmental  philosophy  became  apparent. 

I  see  no  reason  to  retract  any  of  these  statements. 
However,  the  next  quotation  from  the  earlier  paper 
raises  some  interesting  questions. 

While  one  aspect  of  the  military  approach  to  the  develop¬ 
ment  of  the  command  and  control  systems  has  always 
been  to  involve  the  user  as  deeply  as  possible  in  the  sys¬ 
tem  design  phases,  another  aspect  has  been  essentially 
technological.  In  light  of  the  difficulties  described  above, 
some  of  the  military  services  have  sought  to  develop 
general  purpose  data  processing  equipment  and  general 
purpose  computer  programming.  This  technological  ef¬ 
fort  has  been  directed  towards  the  development  of  the 
equipment  and  associated  software  packages  flexible 
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enough  to  permit  a  resultant  system  to  be  transformed 
by  the  user  as  his  needs  changed  and  became  better 
known  through  operational  experience.  Whatever  future 
technological  developments  may  bring,  it  is  clear  that 
current  equipment  and  computer  programming  techniques 
continue  to  impose  severe  difficulties  upon  the  system 
user.  A  major  time-  and  manpower-consuming  effort  is 
still  required  for  the  user  to  achieve  an  operationally  use- 
fid  system  given  the  delivered  hardware  and  software 
packages. 

While  there  are  remote  indications  that  technological  de¬ 
velopments,  such  as  implicit  programming,  may  make  the 
user's  task  somewhat  easier,  one  must  remember  that 
similar  hopes  were  raised  by  technology  a  decade  ago. 

It  is  realistic  to  note  that  no  currently  visible  magic  tech¬ 
nological  wand  appears  likely  to  be  able  to  solve  all  of 
our  command  and  control  developmental  problems. 

Let  us  take  three  somewhat  different  approaches  to 
this  issue: 

1 .  Do  currently  available  software  packages  permit 
us  to  deliver  to  a  user  systems  which  can  readily  (on¬ 
line,  machine-aided)  be  modified  to  create  useful,  func¬ 
tional  software  on  site? 

Direct  experience  with  one  such  package,  JOSS,  and 
knowledge  of  some  of  the  work  sponsored  by  ESD  and 
ARPA  (ADAM,  LUCID,  Lightning,  GENISYS,  and 
GEN1SYS  revisited),  lead  me  to  reassert  that  currently 
available  equipment  and  computer  programming  tech¬ 
nology  continue  to  impose  severe  difficulties  upon  the 
system  user.  A  major  time-  and  manpower-consuming 
effort  is  still  required  for  the  user  to  achieve  an  opera¬ 
tionally  useful  system,  given  the  currently  deliverable 
hardware  and  software. 

2.  Do  current  developments  in  this  technology  indi¬ 
cate  solutions  to  this  problem  which  will  be  available 
by  1968-70? 

The  hopes  that  were  raised  10  years  ago  have  not 
been  fulfilled  in  several  user  commands.  However,  it 
appears  that  on-line,  machine-aided,  “friendly”-to-the- 
user  capabilities  for  on-site  functional  programming 
can  be  reasonably  extrapolated  from  current  technology 
if  the  appropriate  developmental  approach  is  taken. 

3.  Can  we  hope  to  develop  in  the  laboratory  such 
user-oriented  software  for  use  with  the  next  round  of 
data  automation  equipments,  or  must  it  be  done  on  site? 

Any  answer  to  this  question  must  be  based,  at  least 
in  part,  on  opinion.  We  can  hardly  infer  success  from 
past  failures,  but  neither  should  wc  expect  that  labora¬ 
tory-developed  software  will  find  no  external  use.  What 
we  can,  and  I  believe  should,  do  is  to  hedge  our  bets 
by  opting  for  on-site  development  of  specific  portions 
of  the  required  software  package. 

First,  such  on-sitc  development  would  by  its  very 
nature  involve  some  of  the  potential  users  quite  early 
in  the  program,  providing  the  technical  developers 
with  continuing  and  much  needed  user  inputs.  Equally 


important,  the  “education”  of  the  user  would  proceed 
hand-in-hand  with  his  participation.  As  an  outgrowth 
of  this,  we  would  be  in  a  position  to  obtain  meaning¬ 
ful  user  support  for  other  appropriate  developmental 
programs.  Finally,  and  most  important,  user-oriented 
software  would  have  been  developed  in  time  to  take 
advantage  of  the  next  generation  of  data  processing 
equipment.  For  these  reasons,  and  in  connection  with 
the  more  general  and  abstract  laboratory  software  de¬ 
velopments,  it  is  my  hope  that  the  on-site  software 
development  program  described  below  will  be  initiated 
in  time  to  produce  results  by  1968. 

The  pilot  program  should  do  the  following:* 

•  Retain  and  re-emphasize  the  “lead-user”  con¬ 
cept  shown  to  be  so  valuable  in  the  1401-1410 
Air  Force  interim  system  development. 

•  Insure  that  the  newer  technologies,  particularly 
the  potential  capabilities  of  the  “friendly,”  on¬ 
line,  time-sharing  technologies  will  be  imple¬ 
mented  and  explored  in  a  user-command  environ¬ 
ment  and  that  the  successful  results  will  be  made 
available  to  other  Air  Force  commands. 

•  Provide  a  direct  means  for  the  Air  Force  to 
stimulate  and  influence  the  software  and  hard¬ 
ware  manufacturers  to  see  that  such  techniques 
are  developed  with  Air  Force  command  system 
needs  clearly  in  mind. 

•  Make  available  to  the  Worldwide  Military  Com¬ 
mand  System  (WWMCS)  software  and  hardware 
which  has  been  proven  to  be  reliable  and  opera¬ 
tionally  useful. 

The  pilot  program  is  to  provide  certain  capabilities, 
described  below,  which  arc  to  be  available  at  a  selected 
user  command  already  possessing  a  1410  interim  com¬ 
mand  and  control  system.  I  do  not  believe  that  the 
pilot  program  should  attempt  to  duplicate  the  capabili¬ 
ties  of  the  existing  1410  system.  Instead,  it  should  con¬ 
centrate  on  providing  the  following  working  capabili¬ 
ties  at  the  user  site: 

•  On-line**  file  construction  and  data  entry:  to 
permit  individual  (non-programmer)  users  to 
build,  maintain,  and  transform  personnel  files  by 
using  ordinary  logical  categories  and  operational 
terminologies.  In  addition,  in  accordance  with 
standard  operating  procedure  (SOP)  to  be  cstab- 

*1  have  chosen  to  describe  the  characteristics  of  the  pilot 
program  in  terms  of  an  Air  Force  oriented  program  tied  to 
the  already  existing  ADP  phase  of  the  1401-1410  interim  Air 
Force  systems.  However,  with  suitable  change  of  nomencla¬ 
ture,  I  feel  the  program  could  be  made  suitable  for  applica¬ 
tion  within  any  of  the  military  services. 

♦♦Throughout  this  discussion,  a  time-shared  system  is  assumed 
which  permits  several  users  to  have  simultaneous  access  to 
the  available  p/lot  program  capabilities. 
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lished,  this  is  to  permit  individuals  to  use,  build, 
maintain,  and  transform  organizational  files. 
Computer-aided  guidance  in  typical,  though  se¬ 
lected,  operational  language  is  to  be  provided. 

•  On-line  input/output  format  construction;  to  per¬ 
mit  the  individual  users  to  construct  data  entry 
and  output  formats  with  a  freedom  at  least  equal 
to  that  now  available  with  the  RAND  JOSS  sys¬ 
tem.  Computer-aided  guidance  to  facilitate  the 
introduction  and  definition  of  new  operational 
terminology  in  terms  aeecptablc  to  the  machine 
is  to  be  provided. 

•  On-line  file  retrieval  and  data  output:  to  permit 
the  individual  users  to  retrieve  entire  files  or  se¬ 
lected  portions  (assuming  SOP)  and  to  retrieve 
data  previously  entered  by  the  individual  using 
any  of  the  constructable  data  output  formats. 
Computer-aided  guidance  is  to  be  available  to 
facilitate  this  process. 

•  On-linc/off-time  transfer  of  1410  system  files  and 
data:  to  permit  individual  users  to  transfer,  in 
accordance  with  SOP  to  be  established,  com¬ 
puterized  1410  data  for  the  individual  users  in 
the  pilot  program  system.  (This  would  be  two-way 
transfer  in  an  area  to  be  explored  but  not  re¬ 
quired  initially.)  Computer-aided  guidance  is  to 
be  available. 

•  On-line  product  request  aids:  to  permit  individual 
users  to  request  lists  of  pilot  program  products, 
and  to  request  the  instructions  and/or  rules  for 
using  any  product  available  on  the  lists  (a  product 
is  a  software  package  offering  the  capability  to 
accomplish  a  specific  operational  task  such  as 
building  a  given  file,  special  kinds  of  arithmetic 
processing  data,  or  display  construction,  etc.). 
Machine-aided  guidance  is  to  be  available  to 
facilitate  this  process. 

The  first,  second,  and  third  items  are  working  capa¬ 
bilities  to  be  available  for  the  initial  phase  of  the  pilot 
program.  Probably  the  fourth  item  and  especially  the 
fifth  item  will  have  to  be  developed  on  site. 

Other  results  to  be  expected  from  the  pilot  program 
apply  to  the  following  problem  areas: 

•  Standardization. 

•  Technological  development. 

•  Uscr/tcchnical  developer  relationship. 

•  Education  of  the  staff. 

The  recently  held  Air  Force  Command  and  Control 
Users  Symposium!  clearly  indicated  that  guidance  was 
required  for  at  least  the  above  topies.  It  is  possible  to 


tAir  Force  Command  and  Control  Data  Automation  Users 
Symposium,  December  6,  7,  8,  1965,  The  RAND  Corpora¬ 
tion,  Santa  Monica,  California. 


provide  some  general  guidance  as  to  these  and  other 
related  issues,  but  detailed  answers  can  best  be  ob¬ 
tained  only  by  vigorously  pursuing  the  joint  user/ 
technical  developer  pilot  program  described  above.  It 
is  a  matter  of  obtaining  sufficiently  realistic  detail  to 
provide  a  picture  w'hich  is  clear  enough  to  make  basic 
decisions.  For  example,  “How  much  of  what  kind  of 
standardization?”  is  a  matter  that  requires  investigation 
within  the  user  eontext  of  a  pilot  program.  Furthermore, 
the  development  of  standard  techniques  and  procedures 
is  expected  to  be  one  of  the  outputs  of  a  successful  pilot 
program.  These  standardized  techniques  and  proce¬ 
dures,  with  proven  operational  utility  in  at  least  one 
user  command,  would  provide  tested  standard  products 
for  use  and  adaptation  at  other  user  commands. 

As  to  technological  development,  while  we  know  in 
general  the  kind  of  statc-of-thc-art  development  that 
would  be  of  value  to  command  systems,  the  details  arc 
again  lacking.  I  believe  that  they  will  continue  to  be 
lacking,  or  will  be  invalid  if  they  arc  not  obtained  by  a 
carefully  conducted  pilot  program  in  a  user  context. 

All  this  bears  on  the  question  of  the  uscr/tcchnical 
developer  relationship.  It  seems  obvious  that  both  the 
user  and  the  technical  developer  must  be  involved  in 
the  pilot  program.  We  must  provide  the  means  for  the 
technical  development  agency  to  become  aware  of  the 
technical  deficiencies  that  arc  brought  to  the  surface 
during  the  implementation  of  the  pilot  program.  The 
participation  of  the  technical  development  agency  will 
both  accomplish  this  and  help  to  create  the  willingness 
and  capability  to  correct  those  deficiencies.  It  will  also 
help  to  obtain  support  and  funding  for  the  necessary 
development  programs.  It  also  seems  elear  that  the 
technical  developer  and  the  user  still  have  to  learn  to 
work  more  effectively  together.  A  joint  user-technical 
developer  pilot  program  under  the  auspices  of  the  user 
command  would  do  much  toward  meeting  these  needs. 

To  educate  high-level  military  staff  in  the  art  of 
electronic  data  processing,  words  can  only  do  so  much. 
As  was  said  often  at  the  Symposium:  A  produet  is 
worth  a  hundred  thousand  words.  As  early  as  possible, 
the  pilot  program  must  develop  products  which  are 
useful  in  the  eyes  of  at  least  the  battle  staff  of  the  given 
user  command.  I  do  not  think  it  unreasonable  to  assume 
that  products  regarded  as  useful  by  one  battle  staff  will 
be  regarded  as  useful  by  others.  And  if  the  battle  staff 
is  “sold,”  they  arc  the  natural  salesmen  to  the  com¬ 
mander.  Further,  I  believe  that  a  significant  effort 
must  be  made  to  sec  that  the  story  of  the  pilot  program 
is  widely  disseminated  and  that  demonstrations  are  held 
at  the  user  eommand  conducting  the  pilot  program  with 
appropriate  staff  from  other  commands  attending  and 
participating. 

To  sum  up,  it  appears  that  current  technology  offers  a 
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potential  software  development  which  would  permit  on¬ 
site,  on-line  functional  programming.  Such  a  develop¬ 
ment  would  permit  direct  user  construction  and  adapta¬ 
tion  of  man-machine  routines  as  a  normal  on-site  activ¬ 
ity.  Success  in  this  area,  at  least  in  the  sense  indicated 


by  the  pilot  program  and  in  conjunction  with  other 
continuing  laboratory  developments,  would  ultimately 
permit  the  return  of  system  design  to  the  laboratory, 
even  though  the  path  back  to  the  laboratory  leads 
through  an  on-site  development  program. 
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Introductory  Remarks 

Edward  M.  Bennett 
The  MITRE  Corporation 
Bedford,  Massachusetts 

The  initial  drive  to  achieve  an  interactive  and  direct 
working  relation  between  the  many  organizational 
users  of  a  computer  and  the  computer  itself  first  ap¬ 
peared  in  the  design  of  the  SAGE  Air  Defense  Sys¬ 
tem,  now  over  a  decade  old.  Since  then,  with  the  suc¬ 
cess  of  that  venture  behind  them,  advocates  of  direct 
on-line  man-computer  interaction  have  traveled  far, 
preaching  the  likely  benefits  of  on-line  user  tech¬ 
niques  in  business,  engineering,  architecture,  educa¬ 
tion,  and  any  other  serious  human  endeavor  involving 
logical  processes. 

Gradually  over  the  past  decade,  we  have  begun 
to  accumulate  a  diversity  of  experiences  with  systems 
designed  to  emphasize  man/computer  interaction. 


Some  of  the  designers  of  these  systems  are  passing 
into  third  generation  developments,  others  are  actively 
experiencing  the  results  of  initial  system  design  ef¬ 
forts,  and  still  others  are  just  now  entering  the  arena 
of  on-line  interactive  processing  for  the  first  time. 

This  session  planned  to  bring  together  a  group  of 
system  engineers  and  designers  with  some  significant 
current  commitment  to  interactive  processing.  The 
subject  of  discussion  would  have  been  related  to 
matters  of  value,  utility,  economics,  and  the  ad¬ 
vantages  and  disadvantages  of  on-line  interactive 
processing.  The  emphasis  was  to  be  placed  on  the 
appropriate  role  of  such  processing  in  the  variety  of 
services  supporting  the  military  community. 

The  session  participants  were  invited  to  supply 
technical  descriptions  of  the  systems  in  question  to 
be  available  at  the  time  of  the  meeting.  Because  of 
the  postponement  only  one  such  contribution  was 
submitted  in  suitable  form  for  publication. 
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AESOP — A  final  report:  a  prototype  on-line  interactive 
information  control  system 
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INTRODUCTION 

AESOP  (An  Evolutionary  System  for  On-Line  Pro¬ 
cessing)  is  an  experimental  on-line  information 
control  system  realized  in  the  computer  facility  at 
The  MITRE  Corporation.  It  serves  as  a  prototype 
for  a  elass  of  management  or  command  information 
systems  eapable  of  giving  the  user  as  much  on-line 
eontrol  over  system  performance  as  possible.  It 
is  a  display-oriented  system  in  that  a  cathode  ray 
tube  provides  the  primary  means  for  man/machine 
communication.  In  a  system  of  this  orientation,  a 
light  pen  provides  the  primary  means  of  user  eontrol. 
This  eontrol  is  not  limited  to  that  level  of  the  organi¬ 
zation  responsible  for  programming  the  system,  but 
applies  upward  to  the  highest  level  of  executive 
personnel  interested  in  obtaining  direet  access  to 
the  system. 

The  AESOP  system  is  concerned  with  those  cat¬ 
egories  of  problems  that  can  be  characterized  by 
large  amounts  of  data  to  be  stored,  retrieved,  pro¬ 
cessed,  manipulated  and  changed.  The  system  design 
criteria  include  the  following  considerations.  First, 
it  is  intended  for  operation  by  users  whose  knowledge 
of  data  processors  covers  a  wide  spectrum.  The  range 
of  intended  users  includes  top-level  managers,  opera¬ 
tions  analysts  on  their  staffs,  and  system  program¬ 
mers.  Sueh  a  wide  range  of  users  imposes  unique 
requirements.  Some  of  the  system  capabilities  must 
be  simple  to  use  and  should  entail  an  absolute  mini¬ 
mum  of  training.  Other  features  should  allow  the 
staff  operations  analyst  to  make  immediate  and  per¬ 
sonal  use  of  the  data  processor  as  an  analytical 
tool.  The  second  design  consideration  acknowledges 
the  need  to  meet  changing  and  new  requirements 
as  they  are  imposed.  Features  are  incorporated 
which  enable  the  systems  programmer  to  change 
the  system  on-line. 

A  display  and  a  lightpen  provide  the  primary 
means  of  man/maehine  communication.  The  display 
represents  a  very  good  means  of  communication 


beeause  it  ean  be  used  not  only  to  present  the  data 
and  those  commands  required  for  operation  on  the 
data  but  also  instructions  on  how  to  use  the  system 
By  simply  pointing  his  lightpen  at  a  selected  option, 
the  user  tells  the  data  processor  what  to  do. 

The  AESOP-A  prototype*  operates  on  an  IBM 
7030  (STRETCH)  computer  (65 K  memory  with 
64-bit  words)  and  an  IBM  353  disk  storage  unit  with 
a  eapaeity  of  two  million  words.  The  four  user  sta¬ 
tions  each  consist  of  an  on-line  Data-Display  Ine., 
display  eonsole  (DD-13),  a  photoeleetrie  lightpen, 
an  on-line  typewriter,  and  a  Stromberg-Carlson 
3070  medium-speed  printer.  The  AESOP  system  is 
designed  to  take  advantage  of  the  range  of  capabilities 
implied  by  the  7030  eentral  processor  and  the  user 
station  equipment.  While  taking  advantage  of  the 
range  of  equipment  capabilities  implied,  the  design 
philosophy  is  tailored  to  respond  to  the  needs  of 
management  and  command  users. 

The  design  of  a  smooth  man/machine  interface  is 
not  an  easy  task.  Design  considerations  associated 
with  the  development  of  a  display-oriented  system 
will  be  eovered  first,  followed  by  a  review  of  the 
salient  aspeets  of  the  software  architecture  and  a 
description  of  the  user’s  view  of  the  system. 

Design  philosophy 

A  keystone  of  the  AESOP  design  philosophy  is  the 
evolutionary  approach  to  the  development  of  the 
system.  As  a  particular  capability  was  implemented 
and  tested,  the  insights  gained  from  its  use  were 
applied  to  the  development  of  new  capabilities. 
In  particular,  this  approach  was  applied  to  the  design 
of  the  display  programs  in  the  system.  The  display 
formats  and  procedures  for  man/computer  interaction 
ean  proceed  only  so  far  at  the  designer’s  desk.  The 


♦An  earlier  model  of  AESOP  was  reported  in  E.  M.  Bennett, 
E.  C.  Haines,  J.  K.  Summers,  AESOP:  A  Prototype  for  On- 
Line  User  Control.  AFIPS  Conference  Proceedings :  Fall  Joint 
Computer  Conference,  1965,  27,  435-455. 
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detailed  sequence  of  user  actions  and  choices  for  dis¬ 
play  interaction  were  thoroughly  discussed  and  spec¬ 
ified  before  construction  of  the  data  processing 
programs  was  initiated.  However,  as  soon  as  a  capa¬ 
bility  was  operational  and  tested  on-line  at  the  display, 
additional  design  refinements  frequently  became 
obvious. 

One  concrete  example  of  this  concerns  the  develop¬ 
ment  of  skeleton  messages  for  syntax  control  (see 
Syntax  Control  section).  The  system  prevents  syntax 
errors  by  the  user;  if  he  tries  to  make  an  error,  it 
indicates  the  legal  set  of  actions  that  can  be  taken. 
Experience  and  insights  gained  from  operation  of  the 
system  show  that  a  better  design  is  one  that  indicates 
to  the  user  what  set  of  actions  are  legal  and,  if  an  error 
is  made,  merely  provides  an  error  feedback  signal. 

Experiences  such  as  this  led  to  the  following  mode 
of  action  in  software  development.  When  the  essential 
parts  of  a  new  interactive  display  capability  were 
implemented,  the  feature  would  be  tested  and  design 
adjustments  identified.  Then  the  additional  features 
needed  to  complete  the  capability  were  programmed 
and  made  operational. 

Ititra  system  communication 

The  evolutionary  design  philosophy  applied  to 
the  AESOP  system  imposed  a  similar  requirement 
on  the  design  of  the  data  processing  programs  to 
ensure  that  new  capabilities  could  be  easily  incor¬ 
porated.  This  was  solved  by  using  the  AESOP  User 
Language  not  only  as  the  means  of  communication 
between  man  and  the  data  processor  but  also  as  the 
means  of  communication  between  program  modules 
in  the  system  itself.  This  led  to  the  organizational 
structure  of  computer  program  modules  shown  in 
Figure  L  This  figure  shows  how  the  program  modules 
feed  requests  into  the  retrieval  program  for  infor¬ 
mation  from  the  system  data  base. 

The  retrieval  program  was  constructed  to  interpret 
messages  composed  by  the  user  on  either  the  type¬ 
writer  or,  by  means  of  the  communication  tree,  on 
the  face  of  the  display.  Any  other  programs  that 
require  data  from  the  retrieval  program  construct 
messages  identical  to  the  user’s  and  pass  them  to  the 
retrieval  program.  The  retrieval  program  does  not 
know  whether  the  messages  it  processes  are  user  or 
compute  program-generated.  Thus  the  display  pro¬ 
gram  that  generates  a  display  of  30  lines  of  data 
obtained  from  the  data  base  also  composes  and  sends 
30  retrieval  messages  to  the  retrieval  program. 
The  concept  of  intra-system  communication  was 
further  extended  to  permit  many  of  the  lightpen 
actions  to  be  converted  internally  into  AESOP 
User  Language  messages. 


Figure  1 — Simplified  schematic  of  program  module 
organization. 

This  approach  has  several  benefits.  New  programs 
can  be  added  which  simply  call  upon  already-existing 
programs  to  perform  operations  in  a  specific  sequence. 
Consider,  for  example,  the  COPY  program  which 
transfers  data  from  one  data  base  file  to  another.  In 
so  doing,  it  first  uses  the  retrieval  capabilities  of  the 
retrieval  program  to  obtain  the  required  data  and  then 
uses  the  updating  features  of  the  retrieval  program  to 
place  the  data  in  another  file.  The  COPY  program 
itself  does  little  processing  other  than  message  analy¬ 
sis  and  message  sequencing. 

The  technique  of  intra-system  messages  also  con¬ 
tributes  to  the  problem  of  interface  specifications  be¬ 
tween  program  modules.  In  defining  the  User  Lan¬ 
guage,  the  program  input  interface  specifications  also 
are  defined. 

The  simplicity  of  the  intra-system  messages  concept 
is  a  definite  asset.  As  large  computer  systems  evolve, 
their  organization  tends  to  become  too  complex  to  be 
remembered  by  any  one  of  the  system  builders.  As 
a  result,  the  coordination  required  among  the  system 
builders  increases  tremendously.  Any  design  approach 
that  tends  to  maintain  simplicity  within  the  system 
architecture  will  postpone  the  day  when  the  size  and 
complexity  of  the  system  makes  it  difficult  for  the 
system  builder  to  understand  and  alter  the  system  to 
meet  current  operational  requirements. 

System  Languages 

Five  different  programming  languages  were  used  in 
the  construction  of  the  system: 

STRAP  STRETCH  Assembly  Language 

SMAC  STRETCH  Macro  Language 

FORTRAN 


AESOP  71 


TREET*  List  Processing  Language 

TAP  Macro  Language  for  the  List 

Processor 

All  of  the  programs  sit  in  a  FORTRAN  environ¬ 
ment  to  take  maximum  advantage  of  the  FORTRAN 
utility  system  provided  with  the  7030  software  sys¬ 
tem.  The  TREET  list  processor  contains  both  an  in¬ 
terpreter,  especially  useful  for  program  checkout,  and 
a  compiler.  The  programs  are  debugged  on-line  in  the 
interpretive  mode.  When  they  are  accepted  for  opera¬ 
tional  use,  they  are  compiled  to  provide  greater  speed 
of  operation. 

•  MESSAGE  ORIGINATORS  FORM  COMMANDS  THAT  ARE  IN  THE  LANGUAGE 

OF  THE  INTENDED  RECIPIENT 

•  INTERFACE  PROGRAMS  PERFORM  FORMAT  CONVERSION 


STRAP  CODED  TREET  AND 

PROGRAMS  TREET  C00E0 

PROGRAMS 

Figure  2 — Communication  paths  between  conventional 
and  list  processing 

Applications  programs  written  in  TREET  can  ac¬ 
cess  and  modify  the  data  base  in  the  same  manner  as 
other  programs  in  the  system.  A  schematic  representa¬ 
tion  of  the  method  of  communication  between  the 
list  processor  and  the  conventionally  coded  programs 
is  shown  in  Figure  2.  Communication  is  a  two-way 
process  in  that  either  party  may  originate  messages. 
The  originator  composes  messages  in  the  language 
and  syntax  of  the  intended  recipient.  The  message 
then  is  passed  through  the  format  conversion  pro¬ 
grams  (WHYTBOX  and  BLACKBOX  in  the  figure) 
to  convert  these  messages  into  an  internal  format  ac¬ 
ceptable  for  use  by  the  recipient.  This  approach  led 
to  an  economical  interface  in  the  construction  of  a 
hybrid  system  that  employs  both  conventional  and 
list  processing  programs.  The  same  approach  was 
used  to  allow  communication  between  programs 
written  in  the  FORTRAN  language  and  programs 
written  in  other  languages. 


♦The  TREET  language  is  described  by  E.  C.  Haines  in  The 
TREET  List  Processing  Languages;  Bedford,  Mass.,  The 
MITRE  Corporation;  SR- 133,  April  1965. 


Space  allocation 

Several  applications  programs  have  been  overlaid 
on  the  basic  AESOP  system.  The  demands  for  space 
created  by  their  inclusion  could  not  be  satisfied  by 
the  available  core  memory  in  the  STRETCFI  com¬ 
puter.  To  satisfy  this  requirement,  the  memory  was 
logically  divided  into  two  areas  (Figure  3),  one  con¬ 
taining  the  AESOP  executive  and  basic  programs  and 
the  other  the  problem  programs.  Programs,  such  as 
the  retrieval  program,  that  are  used  by  all  problem 
programs  are  stored  permanently  in  core.  All  problem 
programs  are  stored  on  disk.  Whenever  a  specific  pro¬ 
gram  set  is  required  to  process  a  request,  the  pro¬ 
grams  in  core  are  transferred  to  disk  and  the  necessary 
programs  on  disk  are  transferred  into  core. 
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Figure  3 — Core  memory  allocation 
AESOP  data  base  characteristics 

AFSOP  is  a  data  base-oriented  system;  this  char¬ 
acterization  can  be  applied  to  both  its  basic  programs 
and  to  its  applications  programs  since  these  are  pri¬ 
marily  concerned  with  retrieving  data  from  the  data 
base,  performing  operations  upon  it,  and  storing  the 
results  into  the  data  base.  Since  the  AESOP  on-line 
capabilities  are  dependent  upon  the  data  base  struc¬ 
ture,  it  is  essential  to  first  describe  the  organization 
of  the  data  base. 

The  data  base  contains  an  arbitrary  number  of  files 
where  a  file  is  a  named  collection  of  information  about 
similar  entities.  The  entities  in  a  file  are  called  objects; 
each  object  is  identified  by  a  name  or  by  a  number 
which  corresponds  to  its  relative  position  within  the 
file.  The  various  kinds  of  information  which  can  be 
stored  about  an  object  are  called  properties.  Each 
piece  of  information  stored  for  an  object  is  the  prop¬ 
erty  value  of  some  property  in  the  file. 

File  displays  present  requested  data  to  the  user,  and 
are  arranged  in  matrix  form  as  shown  in  Figure  4. 
Each  row  contains  data  for  one  object  in  the  file;  each 
column  contains  values  for  one  property  in  the  file 
and  is  headed  by  the  name  of  the  property  to  which 
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Figure  4 — File  display  format 

the  values  belong.  The  leftmost  entry  in  each  row  is  a 
line  number  which  indicates  the  relative  position  of 
the  object  in  the  data  base  tile.  A  line  number  can  be 
used  in  place  of  the  object  name  in  any  input  message. 
The  entry  to  the  right  of  the  line  number  generally  is 
the  object  name;  it  can,  however,  be  omitted.  The  re¬ 
mainder  of  the  entries  in  each  row  contain  property 
values. 

A  maximum  of  30  rows  of  data  can  be  displayed  at 
one  time.  Many  data  base  files,  however,  contain 
more  objects  than  can  be  displayed  in  30  lines.  Such 
files  are  divided  into  pages  where  each  contains  30 
objects.  A  page  number  shown  in  the  upper  right 
corner  of  the  display  (see  Figure  4)  is  associated  with 
a  given  subset  of  the  objects  in  a  file.  The  first  30 
objects  in  the  file  are  on  page  1 ,  the  second  30  on  page 
2,  and  so  forth.  A  section  number  appears  below  the 
page  number  and  is  associated  with  a  given  subset 
of  the  properties  in  a  file.  The  number  of  properties 
in  a  section  varies;  it  is  determined  primarily  by  the 
size  (length)  of  the  property  values  and  names.  In 
general,  the  properties  in  a  particular  section  are 
logically  related. 

On-line  input  capabilities 

The  AESOP  system  provides  two  different  ways 
for  entering  inputs  on-line:  typewriter  and  lightpen. 
There  is  a  set  of  legal  typewriter  messages  which 
spans  the  complete  range  of  user  actions  (see  the  Ap¬ 
pendix).  Every  system  function  can  be  activated  by 
inputs  from  the  typewriter;  and  many  of  these  can  be 
activated  by  lightpen  inputs. 

In  general,  lightpen  inputs  are  easier,  faster,  and 
less  error-prone  than  the  typewriter  inputs.  Tradi¬ 
tionally  the  lightpen  has  only  been  used  to  make 
simple  requests  or  inputs.  In  the  AESOP  system,  how¬ 


ever,  the  lightpen  is  used  to  compose  complex  file 
modification  and  data  retrieval  messages. 

The  light-pen  is  a  photoelectric  sensing  device  cap¬ 
able  of  detecting  information  on  the  display  scope. 
The  user  focuses  an  aiming  circle  (a  ring  of  light)  on 
information  on  the  display  scope  and  depresses  a 
switch  on  the  handle  of  the  lightpen.  This  user  action 
causes  information  to  be  transmitted  to  the  program; 
the  program  can  then  determine  what  datum  on  the 
scope  face  was  lightpenned. 

The  face  of  the  display  scope  is  subdivided  into  four 
areas:  the  center  or  main  screen  display,  the  upper 
margin  display,  the  left  margin  display,  and  the  right 
margin  display.  The  main  screen  display  area  is  12 
inches  square.  The  margin  areas  arc  12  inches  by  3 
inches.  Data  displayed  in  the  upper  margin  area  con¬ 
tain  information  concerning  legality  and  results  of 
actions  taken  on  other  areas  of  the  display  scope. 
Both  the  right  and  left  margins  contain  commands 
from  which  the  user  selects  the  functions  to  be  per¬ 
formed.  The  upper  right  margin  contains  five  com¬ 
mands,  each  of  w  hich  defines  a  mode  of  operation  for 
lightpen  inputs  at  the  display:  TABULAR,  TREE, 
FILE  MAN  IP,  ERASE,  and  COPY.  The  last  four 
modes  (see  Syntax  Control  section)  are  used  to  com¬ 
pose  messages  with  the  lightpen.  Each  time  the  user 
activates  a  mode,  a  set  of  commands  associated  with 
that  mode  appears  in  the  lower  right  margin  of  the  dis¬ 
play.  For  the  TABULAR  mode,  the  commands 
are: 


Action 
Next  Page 
Previous  Page 
Next  Section 
Previous  Section 
Restore 
Display  Line 

Execute  Stored  Statement 
Print 


Right  Mar  pin  Command 

•  NEXT  PAGE 

•  PREY  PAGE 

•  NEXT  SECT 

•  PRFV  SECT 

•  RESTORE 

•  DISP  LINE 

•  DO 

•  PRINT 


The  commands  in  the  right  or  left  margins  are  exe¬ 
cuted  by  firing  the  lightpen  at  the  command  dis¬ 
played  in  the  margin.  The  DISP  LINE  command  al¬ 
lows  the  user  to  request  an  object  display  for  any  ob¬ 
ject  in  a  file  display.  The  DO  command  allows  a 
user  to  execute  a  stored  statement  in  one  of  the  state¬ 
ment  files  (see  Syntax  Control  section).  The  PRINT 
command  allows  the  user  to  obtain  from  the  Strom- 
berg-Carlson  printer,  a  hard  copy  output  of  any  dis¬ 
play  (except  trees)  shown  on  the  main  screen  area  of 
the  display  scope. 

The  lower  left  margin  contains  commands  to  pro¬ 
vide  a  shortcut  method  by  which  a  user  can  display 
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Page  1,  Section  I  of  a  file  without  composing  the  full 
input  message.  Through  use  of  these  commands  the 
user  can  build  a  list  of  10  file  names  in  the  upper  left 
margin.  This  feature  is  useful  for  both  the  easual  and 
the  steady  users  of  the  system. 

An  example  of  the  display  before  a  list  is  selected 
is  shown  in  Figure  5a.  When  the  lightpen  action  is 
taken  on  SET  FILES  in  the  left  margin,  the  display 
changes  to  present  a  list  of  all  files  in  the  system  as 
shown  in  Figure  5b.  Each  file  name  selected  by  light- 
pen  action  appears  in  the  upper  left  margin  as  shown  in 
Figure  5e.  A  maximum  of  10  file  names  may  be  se¬ 
lected  in  this  manner.  When  the  command,  FINISFI, 
is  lightpenned,  the  previous  file  display  is  restored 
(see  Figure  5d).  Whenever  a  user  now  fires  the  light- 
pen  on  any  file  name  in  this  left  margin.  Page  1, 
Seetion  I,  of  the  selected  file  will  be  displayed. 


Figure  5a — Initial  file  display  and  margin  commands 
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Figure  5b — File  index  from  SET  FILES  action 

Syntax  control 

In  the  TREE  mode  a  communication  tree  is  avail¬ 
able  to  compose  messages  for  requesting  displays. 
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Figure  5c — Consl ruction  of  file  name  index  in  margin 
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Figure  5d — Left  margin  after  FINISH  action 
changing  data  values,  or  for  retrieving  information 
from  the  data  base.  It  is  a  very  useful  device  for  the 
easual  user  since  it  serves  as  a  visual  aid  for  remem¬ 
bering  the  syntax  of  the  AF!SOP  User  Language  and, 
at  the  same  time,  provides  a  look-ahead  feature  to 
guide  the  user  down  the  tree.  1  n  addition,  eertain  prob¬ 
lems  associated  with  typewriter  inputs  are  eliminated. 
Some  of  these  problems  are:  (a)  remembering  message 
formats,  (b)  remembering  the  spelling  of  file  names, 
objeet  names,  and  property  names,  and  (c)  typing 
aeeuraey. 

Assume,  for  example,  that  the  user  desires  to 
change  the  Circular  Error  of  Probability  (Cl  P)  for 
a  weapon  against  a  specific  target  type  (see  Figure  6). 
Lightpen  action  on  the  word  TREE  in  the  upper  right 
margin  of  the  display  initiates  the  tree  mode  and 
eauses  the  communication  tree  to  be  displayed  as 
illustrated  in  Figure  7.  This  tree  contains  the  syn¬ 
tax  of  a  subset  of  the  legal  messages  of  the  AESOP 
User  Language. 
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Figure  6— WEAP-DATA  file  display  with  weapon  code  data 


Figure  7 — AESOP  communication  tree 


The  rules  for  using  the  tree  are  very  simple.  A  user 
can  lightpen  only  those  options  appearing  at  the  top 
of  the  tree.  A  legal  message  is  formed  by  proceeding 
down  a  path  joined  by  vectors.  Initially,  GET  is  shown 
at  the  top  of  the  tree  (Figure  7).  When  the  lightpen 
is  fired  on  GET,  the  display  changes  (see  Figure 
8a),  and  GET  now  appears  above  the  tree  to  form  the 
beginning  of  the  message  the  user  is  constructing.  At 
this  point,  the  user  has  only  one  option;  he  must 
lightpen  FILENAME.  If  the  user  lightpens  any 
other  word  on  the  tree  ERROR  flashes  at  the  top  of 
the  display. 

If  the  user  has  taken  a  legal  action,  the  resulting 
display  (Figure  8b)  contains  a  dynamically  generated 
index  of  the  current  files  in  the  system.  The  user 
chooses  a  particular  file  by  firing  his  lightpen  on  its 
name.  He  is  next  presented  (Figure  8c)  with  three 
options:  RENAME,  which  will  permit  him  to  change 
the  name  or  names  of  one  or  more  entries  in  the 
WEAP-DATA  file;  DISPLAY,  which  will  permit 
display  of  a  particular  page  and  section  of  the  file;  and 


Figure  8a — Result  of  light-pen  action  on  GET 
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Figure  8b — Result  of  FILENAME  action — list  of 
data  base  files 

OBJECTN  AME,  which  permits  the  user  to  choose  a 
particular  file  entry  in  order  to  retrieve  or  change  some 
of  its  properties. 

When  the  lightpen  is  fired  on  OBJECTNAME, 
an  index  of  the  objects  in  the  file  is  generated  (Figure 
8d).  Since,  in  this  example,  the  target  type  to  be 
changed  is  A20L  the  lightpen  is  fired  on  this  label. 
On  the  resulting  display  (Figure  8e),  we  can  choose 
to  rename  this  target-type,  or  to  display  some  or  all 
of  its  properties,  or  to  change  the  stored  value  of 
some  of  its  properties.  When  the  lightpen  is  fired  on 
CHANGE,  an  index  of  the  property  names  is  gener¬ 
ated  (Figure  80*  Since  the  property  we  wish  to  change 
is  the  CEP,  the  lightpen  is  next  fired  on  CEP1. 
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Figure  8c — Next  set  of  tree  options 
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Figure  8d — Result  of  OBJECTNAME  action — list  of  objects 
in  WEAP-DATA  file 
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Figure  8f — Result  of  CHANGE  action — list  of  properties  in 
WEAP-DATA  file 


Figure  9a— WORD  COMPOSER 


Figre  8e — Next  set  of  tree  options 


Figure  9b — 150  in  character  accumulator 
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A  word  composer  (Figure  9a)  is  next  displayed.  The 
word  composer  permits  any  alphanumeric  symbol 
up  to  a  maximum  of  10  characters  to  be  generated  by 
successively  firing  the  lightpen  on  the  characters 
in  the  matrices.  As  each  character  is  fired  upon,  it 
appears  in  the  character  accumulator  between  the 
two  parentheses.  Firing  upon  the  numerals  1,  5,  and 
0  in  succession,  generates  the  value  of  150  (Figure 
9b).  If  a  mistake  is  made  while  a  word  is  being  com¬ 
posed,  a  lightpen  action  upon  the  word,  CANCEL, 
in  the  right  margin  causes  the  character  accumula¬ 
tor  to  be  erased  so  that  a  fresh  start  can  be  made. 
When  the  light-pen  is  fired  on  PROCESS  the  symbol 
in  the  character  accumulator  is  added  to  the  message 
being  composed  at  the  top  of  the  display  (Figure 
9c).  At  this  time  additional  properties  may  be  selected 
in  order  to  change  their  values  or  the  message  may  be 
terminated  by  firing  the  lightpen  on  EOM  (end  of 
message)  in  the  right  margin.  The  resulting  display 
with  the  changed  data  value  is  shown  in  Figure  9d. 
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Figure  9c — Result  of  PROCESS  action 

When  he  is  composing  an  input  message  the  user 
often  requires  the  file  display  in  order  to  see  the  data 
to  be  modified  or  manipulated.  In  such  cases,  another 
form  of  syntax  control  is  used;  the  COPY  mode  is  an 
example.  When  COPY  is  lightpenned,  the  commands 
associated  with  this  mode  appear  in  the  lower  right 
margin  (see  Figure  10a). 

When  one  of  the  first  five  commands  is  light- 
penned,  the  skeleton  message  for  that  COPY  option 
appears  in  the  upper  margin  display  area.  This  mes¬ 
sage  is  a  legal  input  message,  where  a  string  of  dots 
represents  each  parameter  to  be  inserted  by  the  user. 
A  pointer  indicates  which  parameter  the  user  must  1111 


Figure  9d — Result  of  EOM  action — file  display 
with  changed  value 


in  next.  This  message  will  be  used  to  copy  the  data 
from  the  Airfield  file  in  Figure  10a  into  an  individual 
working  file  in  the  system.  Figure  10b  shows  the  mes¬ 
sage  with  the  first  two  parameters  filled,  i.e.,  the  Air¬ 
field  line  numbers  and  properties  specified.  The 
ALLPROPS  option  indicates  that  all  property  data 
is  to  be  copied.  These  parameters  were  filled  in  by 
firing  the  lightpen  on  the  line  numbers  and  on  the 
appropriate  commands  in  the  lower  right  margin.  In 
Figure  10c  the  empty  Work  file  is  shown  with  all 
the  parameter  values  inserted  for  the  skeleton  mes¬ 
sage.  The  Airfield  data  will  be  loaded  into  the  Work 
file  starting  at  line  1,  with  the  properties  of  the  Air¬ 
field  file  mapped  one  for  one  into  the  Work  file.  Fir- 


Figure  10a — Skeleton  message  from  COPY  action 
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ing  the  lightpen  on  EOM  in  the  lower  right  margin 
causes  the  message  to  be  processed  and  produces 
the  results  shown  in  Figure  l()d.  A  private  copy  of 
the  public  data  stored  in  the  Airfield  file  has  been 
made.  Changes  and  alterations  can  be  made  to  this 
data  without  affecting  the  public  data  which  other 
users  may  need. 


that  work  in  a  manner  similar  to  that  illustrated  for 
the  COPY  mode.  The  right  margin  commands  for 
the  FILE  N1ANIP  mode  are  shown  in  Figure  I  la 
and  the  construction  and  result  of  a  SWAP  message 
in  which  lines  of  data  in  the  file  display  are  swapped 
are  shown  in  Figures  I  lb  and  I  1c.  The  COPY  mode 
allows  the  user  to  bring  together  in  one  file  data  from 


The  remaining  lightpen  modes,  ERASE  and  FILE 
MAN  IP,  provide  a  set  of  data  manipulation  actions 
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Figure  1 0b — Parameter  insertion  of  airfield  line  numbers  & 
properties  to  be  copies 


many  different  files.  The  user  may  then  rearrange 
and  format  the  data  into  the  form  for  a  report,  using 
the  FILE  MAN  IP  and  ERASE  modes.  A  hard  copy 
of  the  display  can  be  obtained  on  the  Stromberg- 
Carlson  medium-speed  printer  located  next  to  the 
display  console. 
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figure  lOd — Result  of  FOM  action — -data  transferred 
into  WORK  file 
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Figure  I()c — WORK  file  &  completed  input  message 


Figure  II a — Skeleton  message  &  right  margin  commands  for 
FILE  MANIP  mode 
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It  often  happens  that  a  user  repeats  certain  se¬ 
quences  of  actions  over  and  over.  The  messages  for 
these  actions  can  be  stored  in  the  AESOP  data  base 
as  Stored  Statements  and  executed  by  lightpen  ac¬ 
tions.  The  use  of  Stored  Statements  is  illustrated  in 
Figure  12a  by  means  of  a  subsetting  operation  on  the 
Satellite  file.  Statements  can  be  stored  in  the  data 
base  in  files  named  by  a  single  letter  of  the  alphabet. 
The  B  file  is  shown  in  Figure  12b.  The  statements  are 
stored  in  a  line.  To  look  at  the  complete  contents  of 
a  line,  the  lightpen  is  fired  first  upon  DISP  LINE  in 
the  right  margin  and  then  upon  the  line  number  con¬ 
cerned.  Figure  12c  shows  the  statement,  named 
Select,  that  will  be  operated.  This  statement  contains 
two  system  messages,  separated  by  the  word,  FOM. 
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Figure  I  lb — Completed  SWAP  message  in  upper  margin 


unclossimco 

«cC*jt  r  v,  n 

hOK2 

mC»*! 

Li  1A1C 

covi 

C0l2 

C0t5 

COI* 

COi.3 

•  1 4*141. 40 

l 

00* 

C0V1’«* 

i*T 

lOIC 

ClC« 

■tr  ic 

•  ’•« 
r  ClI  i4i;» 

>002 

M)  l-Ml.’lCS’ 

IRON 

5*1*1 

4*3M 

3* 

0 

l»4SI 

.  co*» 

.1  ,”C1 

444  *-.»  •>  i  - 

:«4i 

1  s 

4*401 

40 

0 

005  *4*141 

’u*tO 

3 ’551 

***•{ 

t?4» 

0 

.  4 

004  :c:rut 

*•41 

52251 

4*251 

500 

0 

«C4*  !4’4 

00’  -414*41 

*.■41 

3*521 

«  •  3  2 1 

3T5« 

0 

ooo  : si 4«4i 

',■*1 

323*1 

3**'C 

323* 

0 

.  tACC  L • 

00*  .lU 

*.■41 

235*1 

3-4*1 

0 

0 

o*o  «r»«4i 

:*4i 

50 '31 

3»3*( 

• 

0 

.  **CC*  4.  IS’ 

*  «r»*4is-4- 

‘,■41 

3*  *1 

4’0?t 

433* 

0 

0*2  <1.«414*4: 

;**i 

332*1 

40  »( 

3 ’50 

0 

4  |  **  •  i-4 

',■41 

2*0*1 

*’231 

0 

0 

0*4  JO" 

'.■41 

34411 

305  I 

3300 

0 

.  $.«4»*J 

o-5  si;»4; 

0  *  S.w’414141 

:*4i 

:»4i 

2*321 

*44 

5235C 

4  .44( 

4*3* 

0 

0 

0 

%(■'  *i.C 

.  •;ts*.ci5 

o”  :o«::4i 

O’* 

'.•41 

2«2tl 

*0341 

0 

0 

»»f.  »*;f 

i 

**« 

- 

4#'4f 

1* 

0 

i(.-  SC 

02* 

»•(.  sc:’ 

•it 

.  set  r ti.es 

*25 

024 

•CS’?*C 

.  ClC4«  CUIS 

*25 

02* 

.  ::si 

.  *:t  ruts 

•2’ 

02* 

.  sc 

•2* 

.  MU’ 

.  **.1SS« 

050 

UNCkASSKlIO 

Figure  11c — Result  of  EOM  action — two  lines  of  data  swapped 


First,  the  Satellite  file  is  displayed,  and  then  all  of 
the  satellites  launched  between  1959  and  1962  are 
selected  and  stored  in  a  new  file.  The  output  file  is 
shown  in  Figure  1 2d  before  the  statement  is  executed. 

The  Stored  Statement  is  initiated  by  a  lightpen  action 
on  the  DO  command  in  the  Statement  file  display 
followed  by  a  lightpen  action  on  the  line  number  of 
the  statement.  When  Select  is  executed  the  selective 
retrieval  program  changes  the  output  file  so  that  it 
duplicates  the  source  file  format  and,  in  addition,  con¬ 
tains  the  results  of  the  subsetting  operation  (see 
Figure  I2e).  Stored  Statements  can  be  created  on¬ 
line,  stored  in  the  data  base,  and  operated  over  and 
over. 
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Procedure  generation 

Other  methods  for  processing  system  data  are  by 
either  executing  or  modifying  established  routines  or 
by  constructing  new  routines  using  the  lightpen  and 
display.  For  these  purposes,  a  routine  called  OAK- 


Figure  12e — Results  of  stored  stalement  operation  in 
the  IF  file 


TREET  is  called  into  operation.  OAK-TREET,  a 
tool  for  on-line  programming  and  debugging,  is  written 
in  TREET,  a  list  processing,  procedure-oriented 
language.  There  are  three  modes  of  operation:  OAK, 
DEBUG  and  SYMBOL.  OAK  is  used  to  construct 
and  execute  procedures;  DEBUG  permits  the  user 
to  display  and  modify  previously  constructed  routines; 
and  the  SYMBOL  mode  is  used  to  modify  individual 
symbols  and  their  properties.  The  lightpen  serves  as 
the  basic  input  device  for  all  three  modes. 

The  displays  are  divided  into  four  areas:  left  margin, 
top  margin,  right  margin  and  center  display  areas  (see 
Figure  13a).  The  center  display  contains  the  work 


Figure  13a — Initial  OAK  display 


Figure  13b — ARITHMETIC  operators  in  upper  right  margin 
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space  for  construction  of  algorithms  and  routines. 
Algorithms  and  routines  called  into  the  work  space 
are  represented  as  tree  structures.  The  commands 
in  the  lower  left  margin  are  used  to  define  a  mode  and 
to  operate  on  the  work  space  as  a  whole.  The  com¬ 
mands  in  the  upper  left  margin  determine  how  subse¬ 
quent  light-pen  actions  in  the  work  space  are  in¬ 
terpreted.  A  lightpen  action  on  one  of  these  com¬ 
mands  causes  the  command  to  appear  in  the  command 
box  area  in  the  top  margin.  The  lower  right  margin 
contains  a  set  of  data  classes.  When  a  lightpen  action 
is  taken  on  a  data  class,  a  set  of  operators  associated 
with  the  data  class  is  displayed  in  the  upper  portion 
of  the  margin.  For  example,  when  a  lightpen  action 
is  taken  on  the  data  class,  ARITHMETIC,  its  set  of 
operators  appears  in  the  top  right  margin  as  shown  in 
Figure  13b. 

A  simple  arithmetic  operation  can  be  used  to  il¬ 
lustrate  the  OAK  mode.  First,  REPLACE  is  fired  on 
by  the  lightpen  and  appears  in  the  command  box  to 
show  that  it  is  the  active  command.  Next,  a  light-pen 
action  is  taken  on  the  word,  EXPONENT,  in  the 
arithmetic  operators  which  causes  the  symbol  to  be 
placed  in  the  data  box  area  in  the  top  margin,  as  shown 
in  Figure  13c.  A  light-pen  action  on  EXP  (expres- 


Figure  13c — EXPONENT  algorithm  in  workspace 


sion)  in  the  work  space  replaces  it  with  EXPONENT. 
EXPONENT  has  two  limbs  to  show  that  it  requires 
two  numerical  expressions  as  parameters  for  its  opera¬ 
tion.  The  data  class,  COMPOSE,  is  fired  on  by  the 
lightpen  to  generate  the  numbers  for  the  algorithm. 
The  right  margin  display  allows  a  user  to  compose  any 
desired  alphanumeric  symbol  by  successive  lightpen 
actions  on  the  characters  (see  Figure  13d).  As  each 
character  is  lightpenned,  it  is  placed  in  a  character 
accumulator  above  the  Word  Composer.  Lightpen 


actions  on  2  and  on  PROCESS  place  the  2  in  the  data 
box  shown  in  Figure  13d.  When  a  lightpen  action 
is  taken  on  one  of  the  nodes  of  the  tree,  the  node  is 
replaced  with  the  2.  In  a  similar  manner,  10  is  con¬ 
structed  and  replaces  the  second  node  of  the  tree  (see 
Figure  13e).  The  procedure  now  constructed  is  “2 
raised  to  the  10th  power.”  The  routine  is  executed 
by  a  lightpen  action  on  EVALUATE  in  the  lower 
left  margin,  and  the  routine  and  its  results  are  printed 
out  on  the  console  typewriter. 


Figure  13d — OAK-TREET  word  composer  right 
margin  display 


Figure  13c — Numerical  values  inserted  in  the  tree 

The  OAK  capability  permits  even  more  compli¬ 
cated  expressions  to  be  constructed.  If  the  entire  tree 
representation  of  a  routine  cannot  fit  on  the  display, 
then  the  workspace  can  be  assigned  to  display  some 
subexpression  of  the  routine.  Figure  I3f  illustrates 
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Figure  I3f-  LOGIC  operators  in  right  margin  & 

FOR  expression  in  workspace 

the  set  of  logical  operators  in  the  right  margin  and  one 
of  these  operators,  FOR,  in  the  workspace.  In  this 
case,  the  list  of  operators  fills  the  right  margin;  hence, 
the  special  operator,  MARGIN  appears  at  the  bottom. 
A  lightpen  action  on  MARGIN  restores  the  data 
class  display. 

The  detailed  structure  of  existing  routines  or  pro¬ 
grams  can  be  examined  and  modified  in  the  DEBUG 
mode.  In  addition,  new  routines  can  be  generated  in 
this  mode.  The  initial  display  is  called  by  a  lightpen 
action  on  DEBUG,  in  the  lower  left  margin.  Changes 
in  the  workspace  area  and  in  the  lower  left  margin 
commands  for  the  DEBUG  mode  are  shown  in  Figure 
1 4.  A  routine  displayed  in  the  DEBUG  mode  is  shown 


Figure  15a — KTIMEMIN  routine  in  DEBUG  mode 

in  Figure  15a.  The  routine,  KTIMEMIN,  converts 
military  time,  e.g.,  1330,  to  time  in  minutes.  Using 
the  word  composer,  the  name  of  the  routine  is  placed 
in  the  data  box  and  displayed  when  a  light-pen  action 
is  taken  on  NAME  in  the  left  margin 
The  left  limb  of  the  tree  shows  the  actual  code  for 
the  routine  written  by  the  programmer.  The  next  limb. 
R,  shows  that  this  routine  is  a  type  R  routine  in  that 
it  requires  a  fixed  number  of  input  arguments.  The 
ARGS  limb  shows  that  this  routine  has  one  input 
argument,  KT.  If  bound  variables  had  been  included 
in  this  routine,  they  would  have  been  placed  on  the 
next  limb  under  PVARS.  To  illustrate  how  a  routine 
is  modified,  a  check  can  be  added  to  the  routine  to 
test  for  a  given  time  greater  than  24  hours.  An  IF 


Figure  14 — Initial  DEBUG  mode  display 


Figure  1 5b — IF  expression  added  to  the  routine 
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statement  is  added  to  the  routine  in  Figure  15b,  by 
means  of  the  ADD-RT  command.  The  ADD-RT 
command  causes  an  expression  to  be  added  on  the 
same  level  and  to  the  right  of  the  node  lightpenned. 
The  IF  expression  in  this  Figure  indicates  the  param¬ 
eters  that  must  be  added  for  its  proper  execution. 

Some  of  the  steps  in  constructing  the  new  check  for 
a  time  greater  than  24  hours  are  shown  in  Figures 
15c  and  15d.  First,  the  conditional  test  phrase  is 


Figure 

15c — Conditional  test 

inserted 
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Figure  15d — True  condition  inserted  &  else  limb  erased 

inserted.  Then  the  next  part  of  the  IF  expression  is 
inserted  to  change  those  input  values  that  are  greater 
than  24.  Finally,  the  ELSE  limb  is  erased,  since  the 
test  is  needed  only  for  a  true  condition.  The  modified 
routine  replaces  the  original  routine  by  means  of  a 
light-pen  action  on  DEFINE  in  the  lower  left  margin. 


An  on-line  debugging  tool  such  as  this  is  a  valuable 
aid  in  reducing  the  elapsed  time  between  the  require¬ 
ment  for  a  change  to  the  system  programs  and  the 
time  when  the  new  or  modified  routine  is  operational. 

Every  TREET  symbol  has  a  unique  value.  This 
value  can  be  either  another  symbol  or  a  list  of  symbols. 
A  symbol  can  also  have  properties  associated  with  it. 
Each  property,  in  turn,  has  a  value  which  is  a  symbol 
or  a  list  of  symbols.  The  SYMBOL  mode  deals  direct¬ 
ly  with  any  symbol  used  in  TREET  and  permits  one 
to  modify  the  symbol  and  its  value  and/or  its  proper¬ 
ties.  The  SYMBOL  mode  is  entered  when  the  light- 
pen  is  fired  on  SYMBOL  in  the  lower  left  margin, 
and  the  workspace  and  the  margin  are  again  altered 
for  the  new  mode  (see  Figure  16a). 


Figure  16a — Initial  SYMBOL  mode  display 


Figure  16b — ARITHMETIC  symbol  expression  in  workspace 

The  contents  of  the  margins  in  the  OAK-TREET 
displays  are  properties  of  system  symbols.  These 
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symbols  control  the  information  displayed  in  the 
margins.  A  user  can  change  the  commands,  data 
classes,  and  operators  available  to  him  by  changing 
the  property  list  associated  with  the  system  symbols. 

For  example,  the  value  of  a  property  of  the  symbol 
ARITHMETIC  can  be  modified  by  expanding  the 
list  members  of  the  property  RIGHT  MARGIN 
(RTMG)  for  the  symbol.  In  essence,  the  right  margin 
for  the  ARITHMETIC  data  class  will  be  changed  to 
include  more  operators.  With  ARITHMETIC  placed 
in  the  data  box,  a  lightpen  action  on  VALUE  causes 
the  symbol  and  its  value  to  be  displayed  in  the  work¬ 
space  (see  Figure  16b),  In  this  case,  the  value  is 
NIL.  The  ARITHMETIC  property  RTMG  is 
brought  into  the  work  space  when  a  light-pen  action 
on  PROPERLY  is  taken  and  after  the  symbol  RTMG 
is  placed  in  the  data  box  (see  Figure  1 6c).  T  his  display 


Figure  16c — RTMG  property  and  its  list  brought 
into  the  display 

now  shows  that  ARITHMETIC  has  a  property 
RTMG  whose  value  is  a  list.  The  fact  that  it  is  a  list 
is  indicated  by  the  dominating  node  of  two  dots.  The 
list  members  are  the  operators  of  the  ARITHMETIC 
data  class  that  were  displayed  in  the  right  margin. 
Now  the  list  is  modified  by  adding  other  members: 
FACTORIAL,  SIN  and  COSINE.  Each  new  mem¬ 
ber  is  placed  in  the  data  box  and  brought  into  the  work 
space  by  the  ANRS  command. 

The  ANRS  command  specifies  that  a  node  is  to  be 
added  as  a  sister  to  the  right  of  the  node  lightpenned. 
After  the  SIN  member  is  added  to  the  list  (see  Figure 
16d),  the  FACTORIAL  and  COS  members  are 
added  but  not  displayed.  Instead,  as  Figure  I6e  il¬ 
lustrates,  a  dot  is  placed  to  the  right  of  the  SIN  node 
to  indicate  that  there  is  not  enough  room  to  display 
the  other  known  members.  Although  these  members 
are  not  displayed  they  are,  in  fact,  included  in  the  list 


Figure  I6d — SIN  added  to  the  list 

since  the  upper  right  margin  for  the  ARITHMETIC 
data  class  shows  all  the  operators  after  a  light-pen 
action  is  taken  on  the  DEFI NE command  in  the  lower 
left  margin  (see  Figure  16e). 


Figure  16e — Result  of  DEFINE  action 
Applications 

This  report  has  described  the  general  purpose 
features  of  the  AESOP  system.  The  availability  of  the 
general  purpose  features  has  eased  the  task  of  em¬ 
bedding  special  purpose  applications  programs  within 
the  system.  A  number  of  applications  program  have, 
in  fact,  been  placed  in  the  system.  For  example,  an 
on-line  linear  program  for  deployment  planning  has 
been  included.  All  of  the  input  parameters  reside  in 
data  files  so  that  the  user  may  use  the  general  purpose 
features,  v/'z.,  file  displays  and  data  updating,  to  view 
and  change  the  input  parameters,  respectively.  When 
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the  linear  program  is  operated,  the  results  are  placed 
in  output  data  files.  The  user  views  the  solution  in  a 
file  display  and  can  obtain  printed  copies  of  it.  If 
the  solution  is  not  satisfactory,  the  user  can  change 
the  input  parameters  and  request  another  solution. 
This  mode  of  operation  puts  the  operations  analyst 
on-line  with  the  computer  in  an  interactive,  problem¬ 
solving  mode. 
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APPENDIX 

AESOP  Users  Language 

Typewriter  Message  Notational  Conventions 
Upper  Case  Words 
Underlined  Words 

Upper  ease  words  which  are  underlined  are  key,  ac¬ 
tion  defining  words  which  must  appear  in  the  actual 
typewriter  input  message. 

Words  Not  Underlined 

Upper  case  words  which  are  not  underlined  are  op¬ 
tional.  Their  presence  or  absence  in  a  typewriter  mes¬ 
sage  does  not  affect  the  meaning  of  the  message.  They 
can  be  included  to  enhance  readability;  they  can  be 
omitted  to  reduce  typing  time. 

Lower  Case  Words 

Lower  ease  words  describe  the  kind  of  information 
to  be  inserted  by  the  operater  at  that  point  in  the  mes¬ 
sage. 

EXAMPLE:  GET  filename  .  .  . 

Punctuation 

Brackets 

Brackets  are  used  to  indicate  portions  of  a  message 
which  may  be  included  or  omitted.  The  presence  or 
absence  of  these  portions  does  affect  the  meaning  of 
the  message. 

Brackets  are  nested.  A  bracketed  portion  of  a  mes¬ 
sage  may  not  be  included  unless  all  bracketed  portions 
of  which  it  is  a  subportion  have  been  included: 
EXAMPLE;  .  .  .  PAGE  number  fSECTION  num¬ 
ber]] 

“SECTION  number”  may  not  be  included  if  “PAGE 
number”  is  not  included. 

Braces 

Braces  are  used  to  indicate  alternative  inputs.  One  of 
the  alternatives  must  be  chosen.  An  alternative  extends 
along  a  horizontal  path  until  the  right  fas  opposed  to 
left)  brace  is  encountered;  one  cannot  switch  paths  with¬ 
in  a  set  of  braces. 


EXAMPLE:  ...  )  ALL  ..  } 

[  property  list  J 

Parentheses 

Parentheses  are  used  in  the  COPY  messages  to  en¬ 
close  the  objeet  specification.  They  must  appear  in  the 
message  and  be  separated  from  data  by  spaces. 
Abbreviations  and  Run-On  Words 
Abbreviations  and  run-on  words,  used  in  message 
descriptions,  are  listed  below  together  with  their  mean¬ 
ings: 

(a)  filename — the  name  of  a  data  base  file 

(b)  objeet — in  all  eases,  either  the  name  of  an 
objeet  (objeet  name)  or  the  number  of  an 
objeet  (objeet  number). 

(e)  objeetspee — specification  of  what  objects  are 
to  be  operated  on.  An  objeet  specification 
can  be  any  combination  of  the  following: 

1.  Objeet  name 

2.  Objeet  (line)  number 

3.  Ascending  range  of  objeet  numbers,  writ¬ 
ten  »#.  Eg.  1  — >20  specifies  objeet  1 
through  objeet  20. 

(d)  ^ — line  number 

(c)  property  list — a  variable  length  list  of  prop¬ 
erty  names. 


Key  Words  Followed  By  Dots 
Two  “phrases”,  the  LIST  phrase  and  the  CHANGE 
phrase,  appear  as  optional  portions  of  several  mes¬ 
sages.  For  the  sake  of  simplicity,  they  are  denoted  by 
LIST  ...  and  CHANGE  ...  except  in  the  messages  in 
which  they  are  the  principle  elements.  They  are  defined 
as  follows: 


LIST 


means  LIST 


list  } 


f  ALL 

\  property  list  J 
CHANGE  ...  means  CHANGE  property  name,  property 
value  ...  property  name,,  property  valuen 

M  es  sage  Ex  plana  tion 

Message  14  enables  one  to  change  the  value  of  a 
single  property  for  several  objects.  The  number  of 
values  following  the  property  name  must  equal  the 
number  of  objects  specified  in  the  objeet  spec. 

Message  16.  The  objects  to  be  operated  on  are 
specified  in  the  objeet  spec.  The  data  to  be  erased  is 
specified  by  one  of  the  following: 

ALL 


specifies  the  objeet  name  and  all 
property  values 

specifies  all  property  values  but  not 
the  objeet  name 
specifies  the  objeet  name  only 
specifies  the  values  of  the  prop¬ 
erties  listed 

Message  30  causes  the  current  versions  of  the  data 
base  files  to  be  saved  on  magnetic  tape. 


ALLPROPS 

OBJECTNAME 
property  list 
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Data  Retrieval  Messages 
Display  Request  Messages 

( I  )  GEL  filename  DISPLAY  [PAGE  number  I  SECTION  number]] 

(2)  DISPLAY  PAGE  number 

(3)  DISPLAY  SEC  1ION  number 

(4)  RESTORE 


5)  QEI  filename  object  DISPLAY  [j  ^  ^  j  J 


ALL 
property  list 


(6)  DiSPLA>  LINE  number 
Hardcopy  Output  Request  Messages 

(7)  Gill  filename  object  LIST  f  AL1  ] 

L  l  property  list  )  J 

Rooster  System  Typewriter  Input  Messages 


(8)  PRIN  1  DISPLAY 
(A>)  G  HI  filename  PRIM 

(10)  KIND  DISTANCE  1  ROM 


latitude  longitude  TO 
filename  object  TO 


[  latitude  longitude  )  I 

l  filename  object  J  \ 

|  latitude  longitude  j  / 

1  [filename]  object  \ 


property  namcn  property  value. 


B.  File  Modification  Messages 

(11)  GET  filename  object  CHANGE  property  name,  property  value, 

1  RENAME  new  object  name]  [LIST....] 

(12)  QEJ  filename  !  RENAME^objea  }  ncW  objcct  namc  ICHANGE....I  [LIST....] 

(13)  GET  filename  RENAME  object,  new  object  name .  object,,  new  object  name, 

(14)  GEL  filename  objectspec  CHANGE  property  name  value,  ...  value. 

(15)  GET  filename  object  }  1  integer,  property  name,  ....  integer,  property  name.  1LIST....1 


C.  Erase  Message 

(16)  GET  filename  objectspec  ERASE 


ALL 

ALLPROPS 

OBJECTNAME 

property  list 


D.  Line  Move  Operations 

(17)  GET  filename  SWAP 


{  LINE  1  |#  AND  £ 

1  LINES  I  l: 


LINES  )  lff-»ff  AND  *— # 
I  LINE  1 


(IS)  QEI  filename  .REORDER  j  )  #^#  AS  ###..  . 


(19)  QEI  filename  INSERT  f{^|s 

E.  Column  Move  Operations 

(20)  GET  filename  objectspec  SWAP 

(21)  GET  filename  objectspec  INSERT 


###  • 


f  BEFORE! 


# 


l  AFTER  J 

propertynamc  AND  propertyname 


COL 
COLS 

COL  1  ,.  (  BEFORE  ]  '  propertyname 

COLS  j  Pr°Pcrt>llst  j  ALTER 


F.  Copy  Message 

r  AIL  PROPS)  (  All  PROPS 

(22)  QQPY  FRQM  filename  (objectspec)  ]  propcrtylist  )  filcnamc  (objectspec)  j propcrtylist 
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G. 


H. 

I. 


(23)  COPY  WITH  OBJECTNAMES  FROM  filename  (objcctspcc)  INTO  filename  (objcctspcc) 

f  ALLPROPS 


(24)  COPY  WITH  OBJECTNAMES  FROM  filename  (objcctspcc) 

f  ALLPROPS  I 
|  propertylist  ) 

(25)  COPY  WITH  PROPNOMF.S  FROM  filename  (objcctspcc) 

(  ALLPROPS  ) 

(  propertylist  I 

\  OBJECTNAMES  AND  PROPNAMES  1 
PROPNAMES  AND  OBJECTNAMES  I 

f  ALLPROPS 


INTO  filename  (objcctspcc) 


INTO  filename  (objcctspcc) 


(26)  CQP-Y  WITH 


propertylist 


f  ALLPROPS  ) 
I  propertylist  | 


FROM  filename  (objectspec) 


ALLPROPS 


[  propertylist  )  ™  filename  (objectspec) 
Selective  Retrieval  Message 


(27)  GET  filename  [objcctspcc]  LF  propertyname 


propertylist 


(LI 

LI  1 

L£Q 

EO 

£1 

■value 

f  AND) 

I  QB.  1  propcrtynan‘e 

LEQ 

EQ 

GI 

•value  . .  . 

tGREQ, 

GREO 

Statement  Message 

(28)  DO  filename  object  [parameter  .  paramctcrn] 

Miscellaneous  Messages 

(29)  DATE  dd-dd 

(30)  SAYE 
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CONTROL  SYSTEMS  COMPATIBILITY 
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Introductory  Remarks 

Thomas  P  Cheatham,  Jr. 
McDonnell  Company 
St.  Louis,  Missouri 


I  am  most  pleased  to  have  this  opportunity  to  par¬ 
ticipate  in  the  Third  Congress  on  Information  System 
Science  and  Technology  and  to  talk  to  you  about  one 
of  my  favorite  subjects  — “Tactical  Command  and 
Control  Systems  Compatibility/'  I  feel  that  it  is  worth¬ 
while  to  devote  some  time  to  this  subject  from  the 
point  of  view  of  the  Department  of  Defense.  “Tac¬ 
tical  Command  and  Control”  is  being  discussed,  and 
programs  developed  for  implementing  systems,  at  all 
levels  in  all  four  of  the  Military  Services.  Of  course, 
today's  tactical  forces  possess  command  and  control 
capabilities.  However,  a  principal  goal  of  our  present 
DOD  effort  is  to  make  command  and  control  more 
effective  by  relating  them  to  “real  time”  in  the  “real 
world.”  The  term  “Tactical  Command  and  Control” 
as  used  here  covers  the  total  capability  of  a  command¬ 
er  to  order  his  forces  and  control  his  weapons.  This 
capability  consists  of  the  staff  and  facilities  at  the  joint 
tactical  command  headquarters,  the  communications 
to  his  component  commanders,  the  staff  and  facilities 
of  subordinate  commands,  and  so  on  down  through 
the  various  echelons  to  the  combat  forces. 

Looking  at  it  in  another  way,  the  tactical  command 
and  control  capability  is  the  sum  of  the  human  contri¬ 
bution,  the  contribution  of  machines,  the  organiza¬ 
tional  structure  in  which  both  work,  and  the  proce¬ 
dures  used  to  perform  the  tasks  which  are,  themselves, 
usually  dictated  by  the  assigned  mission.  In  this  struc¬ 
tural  organization,  men  are  assisted  in  these  tasks  by 
communications  systems,  computers,  input-output 
equipment,  display  devices  (from  simple  “grease  pen¬ 


cils”  to  sophisticated  automatic  displays)  and  all  the 
other  paraphernalia  incident  to  the  accomplishment  of 
the  task.  It  is  this  particular  interrelationship  which 
should  make  our  subject  of  interest  to  a  “Congress  on 
Information  System  Science  and  Technology.” 

Our  primary  area  of  concern  is  related  to  the  words 
“SYSTEMS  COMPATIBILITY.”  Here,  we  are 
concerned  not  so  much  with  the  “Intra-Service”  sys¬ 
tems  compatibility  problem  (which  also  exists)  but 
with  the  “Inter-Service”  systems  compatibility  prob¬ 
lem.  The  problem  of  inter-Sei vice  systems  compati¬ 
bility  is  generated  where  these  “systems”  function 
in  support  of  military  forces  operating  “jointly”  or 
as  components  of  a  Joint  Task  Force.  This  is  the 
type  of  operation  currently  being  conducted  in  Viet¬ 
nam.  Warfare  such  as  this  is  characterized  by  its  highly 
fluid  nature.  Units  are  shifted  rapidly  and  may  be 
committed  to  widely  dispersed  offensive  and  de¬ 
fensive  operations.  The  associated  tactical  command 
and  control  systems  capability  will,  accordingly,  be 
stretched  and  strained  over  the  extended  areas  of  the 
operation. 

Tying  together  the  characteristics  of  complcxibility, 
great  flexibility,  diversity  and  mobility,  all  normally 
inherent  in  the  conduct  of  tactical  operations,  is  no 
small  challenge.  Providing  “Inter-Systems  compati¬ 
bility”  for  the  command  and  control  of  this  compli¬ 
cated  machinery  is  an  area  all  too  often  overlooked 
in  the  normal  preoccupation  with  the  individual  tasks 
of  designing  and  building  the  “individual”  weapons 
systems,  control  systems  and  command  systems 
needed  by  the  tactical  forces. 

In  order  to  view  inter-systems  compatibility  in  its 
proper  perspective,  it  has  been  necessary  to  broadly 
define  what  the  phrase  means  and  then  to  examine 
both  the  operational  and  technical  aspects  of  this  def¬ 
inition.  Since  command  and  control  systems  and  the 
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associated  communications  (commonly  referred  to  as 
C3  systems)  are  basically  “Information  handling  sys¬ 
tems,”  achieving  compatibility  between  them  has  been 
treated  as  a  process  of  accomplishing  an  effective  in¬ 
formation  exchange  between  them.  The  goal  is  to  pro¬ 
vide  the  “commander”  at  each  specific  echelon  with 
the  information  required  to  aid  him  in  arriving  at 
timely  decisions  or  to  accomplish  his  required  com¬ 
mand  functions.  Although  this  may  be  an  easy  state¬ 
ment  to  make  — it  is  a  far  more  difficult  task  to  accom¬ 
plish.  For  example,  compatibility  between  systems 
should  be  included  as  a  requirement  for  system  design 
only  where  necessary  for  intercommunications  in 
joint  operations.  Therefore,  someone  has  to  be 
charged  with  first  analyzing  this  specific  problem  from 
a  “joint  operational  standpoint”  in  order  to  specify 
such  things  as: 

Who  must  communicate  with  whom. 

What  information  must  be  exchanged. 

The  character  of  the  information. 

It  was  also  quite  apparent  that  tactical  C3  systems 
embrace  a  wide  variety  of  systems  and,  therefore, 
compatibility  between  such  systems  in  a  “joint  envi¬ 
ronment”  cannot  have  a  singular  meaning.  In  some 
cases,  systems  perform  similar  functions;  in  other 
cases,  the  functions  performed  may  be  different  but 
related.  An  analysis  shows  that  different  information 
transfer  requirements  can  be  associated  with  each  of 
these  types  of  relationships.  This  implies  a  recogni¬ 
tion  of  the  use  of  different  information  “sources”  and 
“sinks”  within  systems,  as  well  as  the  different  kinds 
of  information  being  transferred  within  the  context  of 
being  “compatible.”  The  appropriateness  of  being 
compatible  becomes  the  key  consideration.  This  in¬ 
volves  such  matters  then  as  timeliness,  detail,  accu¬ 
racy,  and  other  such  descriptors  of  information  con¬ 
tent  to  be  meaningful.  This  problem  was  referred  to 
the  JCS  by  the  Secretary  of  Defense  in  July  1964. 
Captain  Vernon  Micheel  of  the  JCS  Joint  Command 
and  Control  Requirements  Group  will  shortly  tell  you 
where  we  stand  now  in  relation  to  this  particular  prob¬ 
lem  area. 

If  we  look  at  the  JCCRG  action  in  analyzing  the 
“operational”  requirements  for  joint  systems  compati¬ 
bility,  as  the  first  step,  then  we  should  look  at  the 
actions  of  the  JCS  Joint  Standardization  Group  as  the 
second  important  step.  The  JSG  is  primarily  con¬ 
cerned  with  the  technical  aspects  of  inter-systems 
compatibility.  This  aspect  of  the  problem  is  primarily 
communications  oriented  since  the  information  trans¬ 
fers  between  systems  take  place  as  signal  transmis¬ 
sions.  This  area  also  is  not  completely  under  the  pur¬ 
view  of  an  “individual”  system  designer  since  his  sys¬ 
tem  is  always  only  one  “end”  of  any  required  inter¬ 


system  communications  link  involved.  Without  ad¬ 
vance  agreement  between  the  two  individual  system 
communications  points  on  all  of  the  required  technical 
characteristics  (channel  characteristics,  modulation 
techniques,  synchronization  schemes,  character  cod¬ 
ing  and  formatting,  etc.)  not  only  can  the  required 
message  transmission  be  blocked  from  getting 
through;  it  may  not  even  be  able  to  get  started.  Tac¬ 
tical  data  systems  standards,  as  the  name  implies,  rep¬ 
resents  actions  being  taken  to  standardize  on  some  of 
these  variables  when  the  inter-system  exchange  of 
data  between  tactical  systems  is  required.  Lt.  Col. 
Marcus  Jordon  of  the  JCS  Joint  Standardization 
Group  for  Tactical  Communications  and  Control  Sys¬ 
tems  will  cover  this  particular  problem  area,  and  dis¬ 
cuss  the  efforts  of  the  Joint  Standardization  Group 
in  the  related  area  of  ensuring  compatible  tactical  com¬ 
munications  equipment  for  use  in  the  C3  systems. 

A  third  important  step  in  our  efforts  to  solve  the  sys¬ 
tems  compatibility  problem  was  taken  in  March  1966 
when  Deputy  Secretary  of  Defense  directed  the  estab¬ 
lishment  of  the  Joint  Service  Office  for  Advanced 
Tactical  Command,  Control  and  Communications. 
This  office,  in  itself,  has  no  directive  authority.  It  is 
really  a  very  small,  full-time  group  of  Service  techni¬ 
cal  experts  serving  as  a  coordinating  committee  to  as¬ 
sist  the  Director  of  Defense  Research  and  Engineer¬ 
ing.  The  primary  group  activity  is  focused  on  the  re¬ 
search  and  development  programs  of  the  Services  to 
help  achieve  tactical  systems  compatibility  by  seeing 
that  equipments  are  developed  for  use  by  more  than 
one  Service,  where  possible,  and  recommending  for 
consideration  the  initiation  of  new  developments,  if 
necessary  to  improve  Service  operations  and  wher¬ 
ever  possible  to  augment  and  improve  joint  operations. 
Putting  it  another  way,  the  JSO  assists  DDR&E  by 
performing  detailed  reviews  of  the  on-going  or  pro¬ 
posed  Service  R&D  programs  in  the  tactical  C3  area 
in  an  attempt  to  determine  that  unnecessary  dupli¬ 
cation  does  not  exist  and  that  the  Service  considers 
all  possible  technical,  as  well  as  operational  compati¬ 
bility  aspects.  Major  General  Kenneth  C.  Dempster, 
Director  of  Operational  Requirements  and  Develop¬ 
ment  Plans,  DCS/R&D,  Hqs.  USAF,  will  cover  this 
area. 

Finally,  we  have  the  problem  facing  the  Military 
Services  in  the  actual  creation  of  the  tactical  C3  sys¬ 
tems.  Designing  and  building  a  tactical  command  and 
control  “system”  is  a  complex  and  difficult  under¬ 
taking.  Managing  the  birth  and  evolution  of  one  is 
even  more  difficult.  It  was  a  difficult  enough  problem 
when  the  manager  was  faced  with  the  task  solely  of 
creating  a  “system”  that  would  accomplish  the  desired 
goals  of  its  intended  user.  When  we  compound  this 


Tactical  Command  and  Control  Systems  Compatibility  89 


problem  by  adding  in  the  requirements  for  maximum 
use  of  standardized,  readily  supportable  “common” 
components;  the  desires  of  the  “user”  to  obtain  equip¬ 
ment  that  never  fails  in  use,  requires  little  or  no  space, 
consumes  practically  no  power,  weighs  next  to  noth¬ 
ing,  can  be  moved  anywhere,  at  any  time,  by  any 
means,  and  above  all ,  meets  all  the  requirements  for 
inter-system  compatibility  in  a  joint  tactical  environ¬ 
ment,  we  can  see  that  the  Military  Services  have  a 
very  difficult  task.  The  critical  factor  is  not  one  of 
technology  alone,  but  rather  one  of  insuring  the  prop¬ 
er  balance  between  the  “users,”  the  “joint  require¬ 
ments,”  the  “technical  manager,”  and  the  actual 
builder.  General  Earl  E.  Anderson,  Deputy  Chief  of 
Staff  (Research,  Development  and  Studies),  Hqs. 
USMC,  will  cover  this  very  difficult  problem  area. 

In  conclusion,  1  would  like  to  once  again  point  out 
that  the  requirement  for  inter-systems  compatibility 
in  the  tactical  command  and  control  environment  has 
been  well  documented,  aggressively  supported  at  top 


DOD  levels,  and  is  being  addressed  by  many  agencies 
throughout  the  Military  Departments.  It  is,  however, 
a  very  complex  and  challenging  task.  It  cannot  be 
solved  by  “dictum”  alone;  it  will  take  place  by  strong 
and  intelligent  evolution.  Similarly,  solutions  to  the 
broad  problem  of  inter-system  compatibility  are  not 
to  be  found  alone  in  the  specific  development  of  equip¬ 
ments  or  standards.  The  solution  will  come  only 
through  an  understanding  of  the  whole  problem  across 
the  board  and  then  by  the  complete  and  wholehearted 
cooperation  of  all  parties  involved,  not  only  within 
the  military  family,  but  those  representatives  of  in¬ 
dustry  and  the  scientific  community  such  as  repre¬ 
sented  here  today. 

Our  goal  is  clear,  our  task  difficult.  The  final  meas¬ 
ure  of  our  success,  however,  may  only  be  determined 
in  the  field.  Since,  when  all  is  said  and  done,  actual 
inter-systems  compatibility  is  achieved  only  when  the 
respective  systems  are  operationally  deployed  and 
the  required  information  exchange  actually  takes 
place .  We  cannot  afford  to  fail. 


The  role  of  joint  command  and  control  requirements  group 
in  tactical  command  and  control  system  compatibility 


by  Vernon  L.  Micheel,  Captain,  USN 

Joint  Chiefs  of  Staff 
Washington,  D  C. 


INTRODUCTION 

It  has  long  been  recognized  that  a  commander  needs 
the  best  command  and  control  system  he  can  get.  It 
wasn't  surprising  that  the  Services  were  quick  to  apply 
advances  in  information  system  technology  to  improve 
their  command  and  control  systems. 

It  wasn’t  long  before  it  was  recognized  that  there  was 
a  need  for  compatibility  at  the  national  level.  In  Janu¬ 
ary  I960,  the  Joints  Chiefs  of  Staff  (JCS)  undertook  a 
study  leading  to  the  development  of  a  comprehensive 
plan  for  a  joint  military  command  and  control  system. 
In  the  implementation  of  the  plan,  joining  into  a  single 
system  each  of  the  command  and  control  systems  of 
the  Services,  Commanders  in  Chiefs  of  unified  and 
specified  commands  (CINCs)  and  supporting  Depart¬ 
ment  of  Defense  (DOD)  agencies,  there  was  a  serious 
compatibility  problem  between  the  systems  because 
each  system  had  been  developed  to  meet  individual 
requirements. 

It  took  a  bit  longer  to  recognize  that  the  Services 
were  going  their  individual,  separate  ways  on  tactical 
command  and  control  without  any  coordinated  effort 
being  made  to  insure  that  the  tactical  systems  could 
exchange  necessary  information  during  joint  operations. 
1  o  forestall  a  compatibility  problem  in  the  tactical  area, 
the  Secretary  of  Defense,  in  July  1964,  requested  the 
Joint  Chiefs  of  Staff  to  conduct  a  study  which  would 
give  consideration  to: 

•  The  degree  of  commonality  of  operational,  pro¬ 
cedural  and  functional  goals  of  service  systems 
then  under  development. 

•  The  doctrinal,  data  exchange,  and  procedural 
standardization  necessary  to  facilitate  compatibil¬ 
ity  when  these  systems  are  automated. 

•  The  suitability  of  developing  formal  requirements 
for  deployable  joint  task  force  command  and 
control  facilities,  which  would  satisfy  the  re- 

•  quirements  of  the  1968-75  time  frame. 


Joint  command  and  control  requirements  group  in 

tactical  command  and  control  compatibility 

Within  the  Organization  of  the  Joint  Chiefs  of  Staff 
(OJCS),  the  Joint  Command  and  Control  Requirements 
Group  (JCCRG)  under  the  Director,  Joint  Staff,  is 
the  central  point  of  eontaet  for  World-Wide  Military 
Command  and  Control  System  (WWMCCS)  matters. 
The  assignment  of  the  study  on  problems  associated 
with  tactical  command  and  control  systems  was  a  new 
field  for  this  Group  but  was  a  logical  assignment  con¬ 
sidering  the  Group’s  past  experience  in  developing  the 
Concept  of  Operations  for  the  WWMCCS  and  the 
Master  Plan  for  the  National  Military  Command  System 
(NMCS). 

So,  JCCRG,  together  with  Service  representation, 
proceeded  to  develop  a  study  which  would  provide  a 
solution  to  the  problems  the  Secretary  of  Defense  had 
posed.  The  result  was  the  World-Wide  Tactical  Com¬ 
mand  and  Control  Study,  August  1965.  While  this 
study  did  not  provide  the  answer  to  all  the  problems 
posed  by  the  Secretary  of  Defense,  it  was  a  start  in  the 
right  direction  and  formed  the  basis  for  continuing 
programs. 

The  rationale  used  in  approaching  the  compatibility 
problem  was  basically  to  go  back  to  the  operational 
requirements  of  the  user. 

First,  determine,  by  Service,  the  operational  tasks 
and  functions  of  the  user  which  must  be  supported  by 
the  command  and  control  system.  (Land  maneuver, 
close  air  support,  amphibious  operations,  etc.) 

Second,  determine  the  operational  tasks  and  func¬ 
tions  common  to  more  than  one  Service. 

Third,  suggest  appropriate  areas  for  standardization. 
(Terms,  language,  procedures,  etc.) 

Fourth,  determine  the  supporting  command  and 
control  compatibility  requirements.  That  is:  Who  must 
talk  to  whom?  What  information  must  be  exchanged? 
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What  is  the  character  of  the  information?  Note  that 
the  study  is  not  hardware  oriented. 

By  following  this  rationale  it  was  felt  that  the  analysis 
would  support  conclusions  and  recommendations  which 
would  lead  the  way  to  the  point  where  the  technicians 
could  take  over  and  give  technical  character  to  the 
solutions  to  the  problem. 

Early  in  the  development  of  the  study  it  became 
apparent  that  doctrinal  differences  between  the  Services 
for  certain  operational  tasks  might  form  a  block  to  se¬ 
curing  agreed  solutions  in  the  study.  In  order  to  prevent 
this  block  from  occurring,  a  separate  study  was  con¬ 
ducted  by  the  Joint  Chiefs  of  Staff  to  determine  the 
unresolved  (divergent)  issues  between  the  military 
Services  in  the  tactical  command  and  control  area.  The 
study  revealed  only  one  divergent  issue  which  the  Joint 
Chiefs  of  Staff  were  required  to  resolve;  the  concept  for 
airspaee  control.  Specifically,  the  degree  and  manner  of 
control  of  airspaee  over  the  combat  zone.  Once  this 
problem  was  resolved  no  other  doctrinal  issues  pre¬ 
vented  the  accomplishment  of  the  study. 

The  mechanics  used  in  expanding  the  rationale  was 
to: 

a.  Compile  and  compare  the  operational  tasks  and 


functions  of  the  Services  in  joint  operations. 

1  A  total  of  12  broad  operational  tasks  were 

addressed. 

(a) 

Land  combat 

(b) 

Close  air  support  (CAS) 

(e) 

Air  strike/interdiction  (less  close  air  sup¬ 
port) 

(d) 

Air  defense/AAW  (Interceptor) 

(c) 

Air  Defense  (SAM) 

(f) 

Airborne/Air 

(g) 

Air  Reconnaissanee/Surveillance 

(h) 

Arty  fire/Naval  gunfire  support 

(i) 

Air  space  mgt/air  traffic  regulations 

(j) 

Amphibious 

(k) 

Anti-sub  warfare 

(1) 

Search  and  rescue 

2  Each 

of  the  above  tasks  was  evaluated  by  de- 

termining: 

(a)  Where  the  information  (requests,  intelli- 

genee,  orders,  assessment,  etc.)  appears 
geographically  or  organizationally  in  each 
Service  command  and  control  system. 

(b)  The  immediacy  of  the  commander’s  need 
for  information  (“short  term,”  “long 
term”). 

(c)  The  veracity  or  reliability  of  information 
in  terms  of  class  (I  greater  than  0.95%; 
III  about  0.65%). 


(d)  The  degree  of  detail  required  (how  far 
down — platoons). 

(e)  When  information  is  required  (demand 
timing,  e.g.,  3  per  half  hour). 

b.  Determine  the  compatibility  requirements,  that  is, 
provide  solution  to  the  interface  problem  and  informa¬ 
tion  exchange  requirements. 

Further  evaluation  was  made  on  the  above  functions/ 
information  to  determine  the  flow  of  information  among 
Service  forees  operating  jointly.  Diagrams  were  pro¬ 
duced  to  show  “boundary  crossings”  on  the  kinds  of 
information  flowing  to  and  from  the  Service  systems. 

c.  Analyze  the  tactical  command  and  control  system 
in  joint  operations  from  the  data  compiled  in  order  to: 

1.  Determine  which  tasks  are  accomplished  by 
more  than  one  Service  and  establish  need  for  com¬ 
patibility  among  the  Serviee  systems. 

2.  Assess  the  approximate  extent  of  functional 
commonality  which  Serviee  tactical  command  and 
control  systems  would  probably  need  to  possess  in 
order  to  provide  effective  support. 

3.  Recognize  the  operational  requirements  of  the 
supporting  tactical  command  and  control  systems, 
and,  from  guidance  provided  therein,  ultimately  to 
derive  their  technical  design. 

4.  Isolate  similarities  and  dissimilarities  in  terms, 
titles  and  procedures  in  order  to  determine  areas 
suitable  for  standardization. 

The  conclusions  which  followed  from  the  study  were: 

a.  The  operational,  procedural,  and  functional  goals 
of  all  the  Service  tactical  eommand  and  control  sys¬ 
tems  have  a  high  degree  of  commonality. 

b.  The  survey  of  the  operational  information  flow 
across  the  “boundary  line”  between  Service  forces  op¬ 
erating  jointly  and  between  such  forees  and  the  joint 
headquarters  controlling  them  are  first  approximations 
of  user  requirements. 

e.  There  is  a  requirement  for  compatibility  among 
future  tactical  eommand  and  control  systems.  However, 
the  requirement  for  inclusion  of  compatibility  considera¬ 
tions  within  system  design  is  to  be  implemented  only 
in  those  systems  having  a  predictable  requirement  for 
compatibility  with  other  eommand  and  control  systems. 

d.  Many  terms  and  titles  as  related  to  joint  tactical 
operations  should  be  standardized. 

e.  There  is  a  requirement  for  standardization  in  auto¬ 
matic  data  exchange  of  message  formats,  message  trans¬ 
mission  procedures,  and  message  transmission  charac¬ 
teristics. 

f.  It  is  highly  desirable  to  standardize  procedures 
employed  by  different  Services  in  accomplishing  the 
same  tactical  operational  task  in  joint  operations. 

g.  There  is  a  need  for  a  joint  agency  to  be  responsible 
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for  necessary  compatibility,  standardization  and  com¬ 
monality  of  taetieal  command  and  eontrol  systems. 

h.  The  World-Wide  Taetieal  Command  and  Control 
Study  can  form  the  basis  for  development  of  technical 
standardization  criteria  to  insure  compatibility  of  future 
taetieal  command  and  eontrol  systems. 

The  following  recommendations  were  generated  by 
the  eonelusions. 

a.  That  the  World-Wide  Taetieal  Command  and 
Control  Study  be  referred  to  joint  groups  for: 

1.  Development  of  standardized  procedures,  terms, 
and  titles. 

2.  Coordination  of  qualitative  operational  require¬ 
ments  for  tactical  communication-electronic  equip¬ 
ments  when  a  problem  of  compatibility  exists. 

3.  Development  of  standardized  message  formats, 
procedures,  and  transmission  characteristics. 

4.  Development  of  teehnieal  standardization  cri¬ 
teria. 

b.  That  the  Joint  Chiefs  of  Staff  establish  improved 
procedures  for  ensuring  intcr-Serviec  coordination  in 
the  development,  acquisition  and  operation  of  taetieal 
eommand  and  eontrol  systems. 

As  a  result  of  the  approval  of  the  Study  by  the  Joint 
Chiefs  of  Staff  and  the  Secretary  of  Defense,  the  follow¬ 
ing  oeeurred: 

a.  The  World-Wide  Taetieal  Command  and  Control 
Study  was  approved  for  use  by  th^  Services  as  a  state¬ 
ment  of  basie  military  operational  requirements. 

b.  The  Chief,  JCCRG,  was  appointed  chairman  of  a 
Joint  Tactical  Command  and  Control  Procedures  Stand¬ 
ardization  Working  Group  composed  of  Service  repre¬ 
sentatives  to: 

1.  Establish  and  coordinate  a  standardization  pro¬ 
gram  for  operational  procedures,  terms  and  titles. 

2.  Develop  inter-Serviec  coordination  procedures 
for  development,  acquisition  and  operation  of  tac¬ 
tical  command  and  control  systems. 

3.  Review  and,  as  appropriate,  update  the 
WWTCCS. 

4.  Review  and  make  recommendations  for  the  res¬ 
olution  of  unresolved  tactical  command  and  control 
systems  compatibility  problems  brought  to  the  atten¬ 
tion  of  the  Joint  Chiefs  of  Staff. 

e.  The  remaining  recommendations  were  approved 
and  arc  being  implemented  within  the  Directorate  for 
Communications-Electronies,  J-6. 

Continuing  aetions  are  now  going  on  within  the  JCS, 
principle  effort  on  the  operational  side  by  JCCRG  and 
on  the  technical  side  by  Director,  Communieations- 
Eleetronies  Directorate,  J-6.  The  Tactical  Command 
and  Control  Procedures  Standardization  Working  Group 
(TCCPSWG)  from  JCCRG  and  the  Joint  Standardiza¬ 
tion  Group  for  Tactical  Communications  and  Control 


Systems  (JSG/TCCS)  from  J-6  have  been  and  arc 
working  hand  in  glove  coordinating  throughout  the 
proeess  of  arriving  at  standards. 

Under  JCCRG  the  TCCPSWG  developed  the  Inter- 
Service  Coordination  Procedures  which  were  promul¬ 
gated  to  the  Services.  The  TCCPSWG  is  establishing  a 
number  of  Standardization  Field  Panels  (SFP).  These 
panels  meet  for  a  period  of  time  in  the  field,  caeh  panel 
addressing  a  specific  operational  function  area  such  as 
Close  Air  Support,  Air  Strikc/Intcrdiction,  Air  Inter¬ 
cept,  cte. 

The  panels  are  composed  of  officers,  selected  by 
their  parent  organizations  because  they  are  highly 
skilled  in  the  operational  tasks  which  the  panels  are  to 
examine.  They  arc  selected  from  all  the  Services  and 
eertain  unified  and  specified  commands  and  have  a 
working  familiarity  with  their  Service's  doctrine  and 
operational  requirements.  These  officers  have  no  other 
duty  than  to  study  and  analyze  the  operational  task  in¬ 
volved  and  to  assemble  a  document  containing  the  most 
operationally  useful,  mutually  eoneurred  in,  eommand 
and  eontrol  procedures,  terms,  and  titles  that  their 
combined  experience  in  joint  operations  ean  produec. 
With  this  combination  of  professional  competence  so 
precisely  focused  on  a  task,  the  resulting  conclusions 
(derived  from  mutual  agreement)  have  the  highest 
possible  certainty  of  being  acceptable  as  US  standards. 

Completed  SFP  reports  arc  forwarded  to  the 
TCCPSWG  for  review,  approval  and  processing  through 
the  Services  and  Joint  Staff.  Approved  terms  and  titles 
will  be  incorporated  into  the  Joint  Dictionary  by 
Director,  Personnel  Directorate,  J-l,  both  in  the  main 
body  of  the  dietionary  and  in  glossaries  (by  functions) 
to  be  added  to  the  dictionary.  In  addition  a  new  JCS 
Pub — (X)  for  the  present,  will  be  developed  whieh 
will  eontain  all  the  standardized  procedures. 

The  standardized  items  arc  translated  or  transcribed 
by  J-6  into  technical  terms  and  standards  and  placed 
in  JCS  Pub.  10. 

SUMMARY 

In  summary,  the  problem  in  compatibility  was  recog¬ 
nized  by  the  Secretary  of  Defense  and  studied  by  the 
JCS  and  the  Services.  The  resulting  study  did  not 
answer  all  the  problems  but  directed  attention  to  a 
means  of  solving  the  problem.  Assigned  Responsible 
Agcneics  were  specified  in  selected  fields  of  endeavor: 
JCCRG  in  requirements  for  operational  compatibility, 
standardization  field  panels  and  resulting  JCS  publica¬ 
tions;  J-6  in  the  technical  field — to  recommend  stand¬ 
ardization  and  compatibility  criteria  for  taetieal  com¬ 
munications  and  control  systems  to  be  incorporated  into 
JCS  Pubs.  10  and  11;  and  DDR&E  to  coordinate,  re¬ 
view  and  make  recommendations  on  Service  proposed 
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R&D  programs  to  insure  required  inter-Servicc  opera¬ 
tional  compatibility  among  future  tactical  command 
and  control  systems,  in  accordance  with  criteria  estab¬ 
lished  by  the  JCS.  Thus,  the  Joint  Chiefs  of  Staff  have 
developed  programs  which  will  provide  the  tactical 
commander  with  a  compatible  command  and  control 
system.  One  program  will  improve  the  mutual  exchange 


of  information  by  establishing  world-wide  standards 
for  procedures,  terms  and  titles.  Another  will  develop 
criteria  for  the  technical  exchange  of  information  be¬ 
tween  the  various  systems,  both  manual  and  auto¬ 
mated.  The  third  will  insure  Inter-Service  coordination 
in  the  development,  acquisition  and  operation  of  future 
tactical  command  and  control  systems. 


The  joint  standardization  group 

for  tactical  communications  and  control  systems 


by  Marcus  C.  J  ordan,  Lieutenant  Colonel ,  USA 
Joint  Chiefs  of  Staff 
Washington,  D.  C. 

INTRODUCTION 

The  Joint  Chiefs  of  Staff  have  given  increasing  rec¬ 
ognition  to  the  vital  importance  of  compatibility  of 
tactical  command,  control,  and  communications  systems 
and  equipment  during  the  past  several  years.  Com¬ 
patibility  of  equipment  has  always  been  recognized  as 
a  requirement  for  joint  operations.  The  increasing  in¬ 
terdependence  of  forces  engaged  in  military  operations 
and  the  continuing  introduction  of  more  technically 
advanced  systems  and  equipment,  however,  necessitate 
greater  high-level  attention  being  given  to  ensure  com¬ 
patibility  and  the  appropriate  amount  of  commonality. 
The  Joint  Standardization  Group  for  Tactical  Com¬ 
munications  and  Control  Systems  (JSG/TCCS)  is  one 
activity  within  the  Organization  of  the  Joint  Chiefs  of 
Staff  which  has  been  given  a  significant  role  in  ensuring 
compatibility  and  commonality  of  tactical  command, 
control,  and  communications  systems. 

In  April  1  964  the  Joint  Chiefs  of  Staff  were  requested 
by  the  Secretary  of  Defense  to  “develop  standardiza¬ 
tion  and  compatibility  criteria  that  can  be  used  in  the 
selection  of  equipment  for  worldwide  tactical  com¬ 
munications  and  control  systems  of  the  military  serv¬ 
ices.”1 

Already  in  existence  at  that  time  were  numerous 
civilian  and  military  activities  engaged  in  standardiza¬ 
tion  and  compatibility  in  the  C-E  field.  These  included, 
among  others,  the  Military  Communications  Systems 
Technical  Standards  Committee,  the  Military  Commu- 
nications-Electronics  Board,  and  various  groups  under 
each  of  the  Military  Services,  as  well  as  international 
standardization  activities.  Each  of  these  activities  was 
interested  in  compatibility  of  tactical  communications 
and  control  systems  to  some  extent;  however,  there 
was  an  apparent  need  for  the  Joint  Chiefs  of  Staff  to 
establish  criteria  for  standardization  and  compatibility. 
This  need  was  made  more  urgent  by  the  knowledge  that 
compatibility  problems  would  be  tremendously  in¬ 


creased  as  the  newer  and  more  sophisticated  systems 
in  development  were  perfected  for  introduction  into  the 
tactical  forces. 

The  JSG/TCCS  was  created  within  the  framework 
of  the  Joint  Chiefs  of  Staff  to  recommend  these  stand¬ 
ardization  and  compatibility  criteria.  This  Group,  which 
functions  under  the  Director  for  Communications- 
Eleetronies  of  the  Joint  Staff,  was  provided  membership 
from  each  of  the  military  services  as  well  as  the  joint 
staff  and  certain  defense  agencies.  It  was  charged  with 
recommending: 

•  Criteria  to  insure  necessary  compatibility  among 
existing  systems  and  between  existing  and  future 
systems  if  required. 

•  Standards  applicable  to  existing  compatibility  re¬ 
quirements  and  for  the  development  of  future 
systems. 

•  Equipment  commonality  consistent  with  individual 
operational  requirements. 

•  Solutions  to  problems  of  interface  between  non- 
tactical  and  tactical  communications  and  control 
systems  when  required.1 

Excluded  from  consideration  by  the  Group  were 
operational  requirements  and  considerations  of  mission 
and  doctrine. 

In  establishing  this  Group  the  Joint  Chiefs  of  Staff 
recognized  that  one  tactical  communications  and  con¬ 
trol  system  to  meet  the  needs  of  all  Services,  while  it 
might  be  desirable,  was  probably  unrealistic  and  un¬ 
attainable  and  that  the  Services  had  invested  heavily 
in  systems  now  in  use,  that  these  systems  will  continue 
to  be  used  for  some  time,  and  they  will  be  phased-out 
over  a  considerable  period  beyond  that. 

Tactical  Communications  and 

Control  System  Standards 

An  obvious  area  in  which  jointly  agreed  standards 
were  required  was  for  the  exchange  of  real-time  digital 
data  between  automated  air  defense  communications 
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and  control  systems  of  the  various  services.  JCS  Pub¬ 
lication  83  states  that  organizational  arrangements  for 
accomplishing  air  defense  functions  must  provide  com¬ 
patible  electronic  coordination  and  control  means, 
operationally  connected,  when  air  defense  forces  of 
the  various  services  operate  within  a  region.  With  this 
established  requirement  the  first  major  activity  of  the 
JSG/TCCS  was  to  develop  JCS  Publication  10,  Tac¬ 
tical  Communications  and  Control  Systems  Standards. 
This  publication  was  approved  by  the  Joint  Chiefs  of 
Staff  and  the  Secretary  of  Defense  and  published  in 
May  1966. 

The  Tactical  Communications  and  Control  Systems 
Standards  (TACSTANS)  contained  in  JCS  Pub.  10: 

•  Are  developed  for  systems  and  equipment  ap¬ 
plicable  to  functional  areas  in  which  the  need 
for  compatibility  has  been  validated  as  essential 
by  the  Joint  Chiefs  of  Staff. 

•  Are  based  upon  the  philosophy  that  the  interface 
between  tactical  systems  should  exploit  the  maxi¬ 
mum  capability  of  sensors  and  processors  to  pro¬ 
vide  precise  information  interchange. 

•  Utilize  system  characteristics  previously  approved 
for  Service  use  where  these  characteristics  meet 
the  joint  requirements. 


•  Define  interface  standards  in  adequate  detail  to 
guide  the  design  and/or  procurement  of  systems 
and  equipment  so  that  inter-operability  in  the 
field  will  occur  without  modification  or  degrada¬ 
tion. 

•  Include  message  format  standards  designed  to 
support  established  doctrine  and  known  require¬ 
ments.4 

The  standards  in  JCS  Pub.  10  are  designed  to  be 
complementary  to  those  in  Mil.  Std.  188.5  The  data 
elements  and  codes  are  being  further  processed  for 
standardization  under  the  DOD  Data  Elements  and 
Data  Codes  Standardization  Program.  In  the  develop¬ 
ment  of  these  standards,  international  standardization 
requirements,  particularly  those  of  NATO,  which  are 
the  responsibility  of  the  Electronic  Data  Transmission 
Working  Party  (ELDATRAWP),  were  also  given  con¬ 
sideration. 

Chapter  I  of  JCS  Pub.  10  contains  technical  stand¬ 
ards  for  air  defense  and  aircraft  control  which  will  be 
used  by  all  US  tactical  systems  which  have  a  require¬ 
ment  for  inter-Service  data  exchange.  The  tactical  digi¬ 
tal  information  links  (TADILs)  for  which  standards 
are  currently  approved  are  shown  in  Figure  1.  For 
each  of  these  TADILs,  characteristics  have  been  stand¬ 
ardized.  For  example: 


TADIL  A  -  NETTED  DIGITAL  DATA  LINK  UTILIZING  PARALLEL  TRANSMISSION  FRAME 
CHARACTERISTICS  AND  JOINT  STANDARD  MESSAGE  FORMATS  AT  2400  BPS. 
A  —  1  -  INTERCONNECTING  NAVAL  AND  GROUND  ENVIRONMENT  SYSTEMS. 

A  —  2  —  INTERCONNECTING  NAVAL  UNITS. 


TADIL  B  -  POINT  TO  POINT  DIGITAL  DATA  LINK  UTILIZING  SERIAL  TRANSMISSION  FRAME 
CHARACTERISTICS  AND  JOINT  STANDARD  MESSAGE  FORMATS  AT  1200  BPS. 

B  —  1  —  INTERCONNECTING  SAM  WEAPONS  SYSTEMS  AND  GROUND  ENVIRONMENT  AIR 
DEFENSE  SYSTEMS. 

B-  2  -  INTERCONNECTING  GROUND  ENVIRONMENT  AIR  DEFENSE  SYSTEMS. 


TADIL  C  -  TIME  DIVISION  DIGITAL  DATA  LINK  UTILIZING  SERIAL  MESSAGE  FRAME 

CHARACTERISTICS  AND  JOINT  STANDARD  MESSAGE  FORMATS  AT  5000  BPS. 
C  —  I  —  FOR  CONTROL  AND  RECTORING  OF  INTERCEPTOR  AIRCRAFT. 

C  -  2  -  FOR  AIR  TRAFFIC  AND  LANDING  CONTROL. 

C  “3  -  FOR  SURFACE  TARGET  RECTOR  CONTROL. 


Figure  I — Tactical  digital  information  links  (TADILS) 
from  JCS  pub.  10 


•  modulation  and  signal, 

•  frequency  setting,  accuracy  and  stability, 

•  timing, 

•  control  code  structure, 

•  procedural  signals,  and 

•  communications  channel. 

Chapter  II  contains  messages  approved  for  joint 
usage,  together  with  their  characteristics  and  a  state¬ 
ment  concerning  which  messages  are  used  on  which 


TADILs.  There  arc  16  messages  for  use  between  con¬ 
trol  stations  and  between  control  stations  and  surface 
to  air  (SAM)  weapons  systems  on  TADILs  A  and  B, 
and  31  control  and  16  reply  messages  for  use  between 
control  stations  and  aircraft  or  airborne  weapons  sys¬ 
tem  on  TADIL  C.  For  example,  one  standard  message 
is  designed  to  report  the  position,  identity  and  track 
number  of  an  air  track  on  a  coarse  scale.  Included  in 
the  publication  also  are  a  number  of  messages  used  by 
the  individual  services  in  performing  their  air  defense 
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missions  but  which  have  not  as  yet  been  agreed  upon 
for  joint  service  usage. 

Work  is  continuing  within  the  Joint  Standardization 
Group  for  Tactical  Communications  and  Control  Sys¬ 
tems  on  developing  additional  standards  for  applica¬ 
tion  both  to  air  defense  and  other  tactical  functions  to 
be  performed  by  automated  techniques.  Certain  of  the 
messages  now  reserved  for  use  by  individual  services 
arc  also  being  processed  for  joint  approval.  Standards 
for  a  low-speed  TAD1L  (TADIL-D)  have  been  de¬ 
veloped  and  are  being  processed  for  approval  of  the 
Joint  Chiefs  of  Staff. 

The  results  of  the  Worldwide  Tactical  Command  and 
Control  Study  conducted  by  the  Joint  Command  and 
Control  Requirements  Group  of  the  Organization  of  the 
Joint  Chiefs  of  Staff  and  the  follow-on  effort  to  stand¬ 
ardize  procedures,  formats,  terms,  and  titles  for  use  in 
joint  operations  will  provide  the  basis  for  the  continuing 
development  of  JCS  Pub.  10.  With  the  operational  re¬ 
quirement  for  compatibility  as  well  as  the  amount  and 
kind  of  information  that  requires  interchange  estab¬ 
lished,  the  /JSG/TCCS  will  be  able  to  effectively  de¬ 
velop  the  standardized  rtansmission  characteristics  and 
message  formats  and  transmission  procedures  to  insure 
compatibility  of  automated  systems  where  compatibility 
is  required.  The  further  development  of  these  standards 
is  obviously  a  major  under-taking  and  will  represent  a 
significant  part  of  the  activities  of  the  /JSG/TCCS 
for  several  years  even  after  all  required  input  infor¬ 
mation  has  been  developed,  approved,  and  received  by 
the  Group. 

Tactical  Communications  Planning  Guide 

The  other  major  activity  of  the  JSG/TCCS  is  con¬ 
tinuing  development  and  maintenance  of  JCS  Publica¬ 
tion  11,  Tactical  Communications  Planning  Guide. 
This  guide  is  a  direct  outgrowth  of  a  comprehensive 
study  of  tactical  communications  made  by  the  Joint 
Chiefs  of  Staff  in  1965  at  the  request  of  the  Deputy 
Secretary  of  Defense.  The  main  objectives  of  the  study 
were  to; 

•  Ascertain  and  recommend  means  of  increasing 
commonality  and  compatibility  of  communica¬ 
tions  equipment  used  by  tactical  forces. 

•  Recommend  means  of  decreasing  variety  and 
quantity  of  tactical  communications  equipment. 

•  Recommend  means  of  limiting  crowding  of  the 
radio  frequency  spectrum. 

•  In  each  of  the  above  to  give  due  consideration 
to  dollar  and  manpower  costs  involved.6 

The  study  was  conducted  over  a  six-month  period 
by  a  study  group  of  approximately  20  personnel  from 
the  Services,  the  joint  staff  and  defense  agencies  under 
the  chairmanship  of  Major  General  W.  T.  Smith,  Dep¬ 


uty  Director,  Communications-Elcctronics,  Organiza¬ 
tion  of  the  Joint  Chiefs  of  Staff.  A  part  of  the  report 
of  the  study  group  addressed  policy  and  procedural 
matters  affecting  commonality  and  compatibility.  It  in¬ 
cluded  discussions,  conclusions  and  recommendations 
covering 

•  communications  doctrine, 

•  planning  and  programming, 

•  operational  requirements, 

•  defense  standardization  program, 

•  procurement, 

•  radio  frequency  spectrum,  and 

•  communications  security. 

A  second  part  of  the  report  of  the  study  group  was 
entitled  Tactical  Communications  Planning  Guide.  This 
part  was  designed  to  be  the  first  edition  of  a  document 
that  would  be  maintained  and  impro\ed  upon  by  the 
Joint  Chiefs  of  Staff  in  the  future.  The  purpose  of  this 
publication  is  to  provide  guidance  to  the  Services  in  tac¬ 
tical  communications  equipment  planning  and  to  assist 
the  Office  of  the  Secretary  of  Defense  in  reviewing  tac¬ 
tical  communications  matters.  The  concept  of  this  guide 
was  aproved  by  the  Joint  Chiefs  of  Staff  and  the  Secre¬ 
tary  of  Defense,  who,  in  January  1966  approved  the 
Tactical  Communications  Planning  Guide  as  a  general 
guide  for  use  in  planning  future  tactical  communications 
and  requested  that  it  be  updated  for  use  in  appropriate 
1966  program  reviews.7  The  updated  edition  was  ap¬ 
proved  by  the  Joint  Chiefs  of  Staff  for  promulgation 
as  a  JCS  publication  and  forwarded  to  the  Secretary 
of  Defense  in  April  1966. 

The  guide  contains  four  major  chapters  covering 
Categories  of  Tactical  Communications  and  Compati¬ 
bility  Requirements,  Current  Equipment  Compatibility, 
Operational  Requirements,  and  Technological  Com¬ 
patibility  Objectives.  It  should  be  emphasized  that  this 
document  is  concerned  at  this  time  solely  with  tactical 
communications  and  not  with  any  of  the  multitude  of 
other  aspects  of  tactical  command  and  control.  Even 
within  the  area  of  tactical  communications,  coverage 
is  selective  in  the  chapters  dealing  with  operational  re¬ 
quirements  and  current  equipment. 

Operations  conducted  by  joint  forces  involve  various 
combinations  of  closely  interrelated  operational  tasks 
and  numerous  functional  applications  of  communica¬ 
tions  necessary  for  task  accomplishment.  Compatible 
communications  equipment  must  be  made  available 
wherever  intercommunication  is  required.  The  require¬ 
ment  for  compatibility  can  be  stated  in  at  least  three 
ways:  in  terms  of  the  operational  task,  in  terms  of  the 
communications  function,  and  in  terms  of  the  technical 
characteristics  of  the  equipment.  A  technical  categoriza¬ 
tion  was  considered  to  be  most  appropriate  for  the 
guide;  accordingly,  Chapter  I  identifies  a  number  of 
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technical  categories  within  which  compatibility  is  re¬ 
quired. 

In  the  Worldwide  Tactical  Command  and  Control 
Study,  developed  by  the  Joint  Command  and  Control 
Requirements  Group,  12  broad  operational  tasks  arc 
identified  and  requirements  for  intercommunications 
between  units  engaged  in  performing  these  operational 
tasks  specified.  Based  upon  this  information,  the  tech¬ 
nical  categories  of  communications  compatibility,  and 
a  similar  categorization  of  the  applications  which  com¬ 
munications  perform,  a  series  of  matrices  were  de¬ 
veloped  which  describe: 

•  Who  must  intercommunicate  with  whom  in  per¬ 
forming  each  broad  operational  task. 

•  Why  they  must  intercommunicate  (i.c.,  for  what 
communications  functional  application). 

•  How  the  intercommunication  can  take  place 
(which  of  the  technical  categories  of  communica¬ 
tions  compatibility). 

•  When  intercommunication  is  required  (in  sup¬ 
port  of  which  broad  operational  task). 

Figure  2  is  an  example  of  one  entry  from  one  of 
these  matrices. 

LAND  COMBAT  OPERATIONS  MATRIX 


D  —  HP  558 

E  -  VHF  FM  20  -  76  MC/S 
R  -  RADIO  RELAY 


NUMBERS  SHOW 

COMMUNICATIONS  FUNCTIONAL 
APPLICATIONS 

5  -  COMD  ELM  AND  OPN  CEN  COMM 
7  -  TAC  AREA  SPT  COMM 

COMMUNICATIONS 


Figure  2 — Matrix  example  from  JCS  pub.  1 1 


In  Chapter  II  a  total  of  over  700  major  items  of 
tactical  communications  equipment  cither  in  the  active 
inventory  or  in  development  with  operational  use  in¬ 
dicated  in  the  near  future  are  categorized  by  technical 
categories  of  communication  compatibility.  Each  of 
the  more  than  100  Current  Equipment  Compatibility 
Sheets  lists  the  compatible  equipment  in  a  category  and 
gives  an  indication  of  the  extent  of  commonality.  Fig¬ 
ure  3  is  an  example  of  one  of  these  sheets. 

In  Chapter  III  the  operational  requirements  docu¬ 
mented  by  the  Services  in  Qualitative  Material  Re¬ 
quirements,  Specific  Operational  Requirements,  etc., 
are  categorized  in  a  manner  similar  to  that  employed 
for  current  equipment,  and  analyzed  to  determine  the 
extent  of  joint  interest  in  each  requirement.  About  100 
requirements  were  documented  by  the  Services.  10 


EXTRACT  CURRENT  EQUIPMENT  COMPATIBILITY  SHEET  NO.  D14 


NT  SSB  RADIO  TRANSCEIVER,  VEHICULAR 


NOMENCLATURE 

FREQUENCY 

MODULATION 

RECOMMENDED 

REPLACEMENT/ 

STANDARD 

AH/GRG—106 

2-30  MC/S 

A  1  A3,  A 3a,  A 9c 

STD 

AH/MRC— S3 
AN/TRC— 73 

2-30 

A 1,  A3a,  F  1 

STD 

STD 

AH/MRG—95 

2-30 

A 1,  A3a 

5N/VSC— 2 

AN/PRC-47 

2-12 

A2.  A3* 

NOTE  1 

NEW  DEVELOPMENT: 

AN/VSC-2 

2-30 

A  lt  A3,  A3a.A9c.FI  PLD  STD 

NOTE  i:  AN/PRO— 47  IS  A  100W 

STD  FOR  U.S.  MARINE  CORPS. 

AH/PRG—62  WILL  BE  A  20W  REPLACEMENT  FOR  U.  S.  ARMY, 

Figure  3 — Example  of  current  equipment  compatability 
sheet  from  JCS  pub.  1 1 

per  cent  of  these  were  considered  to  be  only  of  interest 
to  a  single  service;  the  remaining  90  per  cent  were 
considered  as  joint  requirements  (a  total  of  34)  based 
on  expressions  of  interest  by  one  or  more  services  in  a 
requirement  documented  by  another  service.  For  each 
of  these  joint  requirements  a  cover  sheet  incorporates 
an  analysis  of  the  degree  of  joint  interest  in  the  re¬ 
quirement  and  a  summary  of  future  action  to  be  taken 
on  the  requirement.  Because  of  the  rather  lengthy 
process  required  for  complete  inter-scrvice  coordina¬ 
tion  of  requirements,  the  joint  requirements  in  the 
present  version  of  the  guide  arc  listed  as  “interim”; 
however,  action  is  under  way  to  complete  the  coordina¬ 
tion  and  remove  the  interim  designation. 

Chapter  IV  contains  technological  objectives  which 
are  considered  important  to  achieving  optimum  com¬ 
patibility  of  tactical  communications  in  the  future. 
These  objectives  are  of  two  types — general,  which  are 
applicable  to  more  than  one  of  the  technical  categories, 
and  specific,  which  are  applicable  to  only  one  category. 

Much  development  work  remains  to  be  done  in 
each  of  the  areas  addressed  in  JCS  Pub.  1 1.  In  addition, 
continuing  maintenance  will  be  required  as  technology 
and  requirements  change.  These  functions  of  develop¬ 
ment  and  continuing  maintenance  will  be  accomplished 
through  the  mechanism  provided  by  the  JSG/TCCS. 

Reorganization  of  Joint  Standardization  Group  for 
Tactical  Communications  and  Control  Systems 
The  Joint  Chiefs  of  Staff  have  recently  approved  a 
reorganization  of  JSG/TCCS.  The  purpose  of  this  re¬ 
organization  is  to  provide  for  continuing  development 
and  maintenance  of  JCS  Pubs.  10  and  11  and  to  permit 
the  Group  to  more  readily  recommend  equipment  com¬ 
patability  criteria  and  technical  and  procedural  stand¬ 
ards  for  tactical  command,  control,  and  communica¬ 
tions  systems  in  support  of  joint  operations.  The 
permanent  organization  and  membership  of  this  Group 
is  now  as  indicated  in  Figure  4. 
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note:  ad  hoc  working  groups  formed  as  required,  each  working  group 

REPORTS  TO  ONE  OF  THE  SUB— GROUPS  SHOWN. 

Figure  4 — Permanent  organization  of  joint  standardization 
group  for  tactical  communications  and  control  systems 


In  addition  to  the  permanent  organization  shown  a 
number  of  working  groups  arc  actively  engaged  in 
working  on  individual  actions. 

In  general  terms,  the  objective  of  Tactical  Communi¬ 
cations  and  Control  Systems  Interface  Standards 
Sub-Group  is  to  define  standardization  and  compatibil¬ 
ity  criteria  for  information  exchange  between  tactical 
communications  and  eontrol  systems  of  the  military 
services.  The  work  of  this  sub-group  will  be  largely 
devoted  to  continuing  the  development  of  JCS  Pub. 
10,  Tactical  Communications  and  Control  Systems 
Standards ,  and  to  related  international  standardization 
efforts.  The  Taetieal  Communications  Categories  and 
Requirements  Sub-Group  and  the  Tactical  Communi¬ 
cations  Equipment  and  Technological  Compatibility 
Objectives  Sub-Group  both  have  basic  objectives  re¬ 
lated  to  the  further  development  and  maintenance  of 
JCS  Pub.  1 1,  Tactical  Communications  Planning  Guide. 
Over-all  guidance  and  coordination  is  provided  by  the 
parent  group. 
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During  the  past  two  and  a  half  decades  improved 
capabilities  in  electrical  communications  and  in  auto¬ 
mated  data  processing  have  received  extensive  ap¬ 
plication  in  support  of  military  operations. 

At  the  risk  of  providing  to  the  information  system 
scientist  merely  another  glimpse  of  the  obvious,  this 
paper  is  intended  to  review  some  of  the  facets  and 
consequences  of  applying  current  technology  in  these 
fields  to  the  solution  of  long  standing  military  opera¬ 
tional  problems. 

Information  has  always  been,  and  continues  to  be, 
the  primary  tool  of  military  command.  Information 
serves  as  the  basis  for  the  commander's  decision¬ 
making,  and  as  the  means  for  conveying  his  intentions 
to  his  command  and  to  others. 

Consequently,  the  acquisition  and  dissemination 
of  information  have  preoccupied  military  com¬ 
manders  as  long  as  they  have  existed. 

Information,  in  the  context  of  the  military  opera¬ 
tional  command  environment,  is  selected,  evaluated 
data,  relevant  to  a  specific  situation  in  point  of  time¬ 
liness  and  content.  Data  are  not  information  until 
these  data  are  perceived  by  the  commander  to  be 
valid,  and  of  value.  Furthermore,  data  cannot  be 
transformed  into  information  until  the  problem  to  be 
addressed  has  been  defined  and  classified.  Until  this 
happens,  no  one  can  know  what  are  the  “pertinent 
facts";  one  can  only  know  data.  Definition  and  classi¬ 
fication  determine  which  data  are  to  be  considered 
relevant.  Analysis  and  perception  determine  which 
data  are  to  be  considered  of  value.  In  the  military 
operational  environment,  the  means  or  method  of 
transferring  these  data  is  the  classical  "communica¬ 
tions  system." 

With  information  — that  is,  relevant  valid  data  — es¬ 
sential  to  military  command,  this  component  sub¬ 
system  for  the  transfer  of  data,  plus  the  mechanical 
and  electrical  methods  available  to  select,  correlate 
and  display  useful  data  for  analysis,  is  essentially  the 


military's  equivalent  to  the  scientist's  "information 
system." 

An  effective  military  communications/information 
system  must  keep  not  only  the  commander  but  all 
levels  of  a  command  adequately  informed  of  changes 
in  the  military  situations  which  affect  them.  To  ac¬ 
complish  this,  personnel  of  the  system's  organiza¬ 
tion,  both  data  handlers  and  data  originators,  must 
know  what  data  to  collect  and  to  process,  and  the 
data  users  must  be  able  to  delineate  their  require¬ 
ments  for  information. 

The  effectiveness  of  a  communications/information 
system  in  supporting  a  command  depends  not  upon 
the  amount  of  data  flowing  in  the  system,  but  upon 
the  amount  of  relevant  data  the  commander  receives 
and  the  degree  to  which  the  system  can  transfer  that 
relevant  data  to  potential  users. 

Historically,  military  communications  have  been 
virtually  synonomous  with  the  exercise  of  authority. 
Further,  from  the  commander's  point  of  view,  his 
organization  is  primarily  a  communications  network. 

One  can  conjecture  that  primitive  military  opera¬ 
tions  were  successfully  conducted  by  a  commander 
who  relied  solely  upon  his  own  sensory  inputs,  re¬ 
called  relevant  experiences  from  memory,  performed 
mentally  the  comparative  analyses  required  under 
the  circumstances,  and  communicated  his  intentions 
or  his  will  directly  to  subordinates  (and  to  adver¬ 
saries)  by  invoking  appropriate  action. 

Proxy  observations  and  the  necessity  to  extend  the 
scope  of  the  primitive  military  commander's  influence 
dictated  a  need  for  assistance  in  achieving  the  desired 
flow  of  information,  and  the  role  of  the  military  com¬ 
municator  was  born. 

As  the  complexity  of  the  military  commander’s 
operations  further  increased,  two  types  of  delegation 
occurred:  staff  subordinates,  to  assist  the  commander 
in  performing  his  functions  of  data  correlation  and 
analysis,  and  line  subordinates  (i.e.,  subordinate 
commanders)  to  assist  the  commander  in  the  com¬ 
munication  of  his  intentions. 
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To  provide  for  mutual  understanding,  to  standard¬ 
ize  behavior,  and  to  minimize  information  exchange 
requirements,  a  third  delegation  of  a  sort  occurred  in 
the  emergence  of  predetermined  and  prepositioned 
policies,  plans,  doctrines,  and  procedures,  to  govern 
the  performance  of  both  the  staff  and  the  line  hier¬ 
archies,  in  the  absence  of  direct  communications  with 
the  commander. 

As  military  weapon  ranges  and  lethalities  increased, 
as  time  available  for  response  to  changing  situations 
seemed  to  dwindle,  as  mobility  and  methods  of  com¬ 
municating  improved,  as  political,  economic,  and 
social  implications  of  military  action  became  more 
obvious  and  critical,  significant  improvements  in 
mechanical  and  electrical  data  handling  capabilities 
were  sought  by  the  military  commander  to  assist 
him  and  his  staff  and  line  subordinates  in  coping  with 
the  increasing  flow  of  data  and  the  increased  need 
for  more  information  associated  with  more  complex 
military  operations. 

The  communications/information  systems  which 
have  evolved  in  support  of  the  command  structure 
in  military  organizations  differ  among  each  other  in 
technical  configuration  and  reflect  subtle  differences 
in  military  roles  and  missions,  in  the  attitudes  of  the 
military  commanders  served,  and  in  the  manner  in 
which  the  command  echelon  is  structured. 

These  differences  fall  generally  into  one  of  three 
categories  or  types,  probably  best  described  by 
analogy.  One  type  of  system  might  be  characterized 
by  a  “public  utility"  concept.  Responsibility  for  the 
service  and  support  thus  provided  to  the  command  is 
shared  between  the  supporting  organization  and  the 
using  command  itself.  The  line  of  demarcation  of 
this  responsibility  falls  roughly  at  the  point  data  enter 
into  and  leave  this  “public  utility"  type  system. 

Its  advantages  are  obvious.  The  “public  utility" 
aspect  of  its  operation  is  governed  by  relatively  con¬ 
cise  technical  principles  and  its  management  and  oper¬ 
ators  can  be,  and  are  usually,  technically  oriented. 
Its  behavior  can  be  carefully  structured  and  its  techni¬ 
cal  performance  predicted  in  the  presence  of  ap¬ 
propriate  standards.  It  is  obviously  economical  of 
resources. 

The  major  disadvantages  of  this  first  type  of  system 
are  also  obvious.  The  “public  utility"  aspect  of  the 
system  and  its  personnel  rely  upon  advance  planning, 
policy,  doctrine  and  procedures;  there  is  little,  if 
any,  flexibility,  obligation  or  capability  to  react 
promptly  to  the  dynamics  of  the  using  command’s 
changing  requirements,  Moreover,  responsibility  for 
“writer  to  reader"  delivery  of  data  is  vested  in  more 
than  a  single  organization. 


A  second  configuration  employs  the  same  fore¬ 
going  “public  utility"  concept  for  trunking  and  inter¬ 
switch  networks,  but  with  a  significant  organizational 
difference.  An  individual,  usually  a  direct  representa¬ 
tive  of  the  commander,  is  designated  the  command 
communicator  and  assigned  the  primary  responsi¬ 
bility  for  “writer  to  reader"  delivery  of  data.  He  is 
technically  competent  to  interface  with  the  “public 
utility"  system  into  which  the  command  has  access. 
In  addition,  he  controls  both  the  terminal  portion  of 
the  system  serving  the  commander  and  the  staff, 
and  may  pre-empt  a  predetermined  portion  of  the 
“public  utility"  network  resources.  He  maintains  the 
close  liaison  with  the  commander  and  staff  necessary 
to  observe  and  modify  system  performance  to  match 
the  dynamics  of  the  command  situation,  and  when 
possible,  to  foresee  and  act  upon  changing  conditions 
before  they  occur. 

Advantages  include  improved  “writer  to  reader" 
delivery  time.  Studies  have  shown  that,  other  con¬ 
ditions  of  the  communications/information  system 
being  fixed,  the  greatest  potential  improvement  in 
message  handling  times  can  be  achieved  in  the  term¬ 
inal  portions  of  the  communications/information 
system.  Therefore,  with  the  manager  of  these  data 
handlers  reporting  directly  to  the  commander,  the 
performance  of  the  terminal  portion  of  such  systems 
appears  to  be  the  critical  feature  which  determines 
the  advantage  of  this  configuration. 

The  second  configuration  also  has  the  same  dis¬ 
advantage  as  the  first  configuration,  if  to  a  lesser 
degree,  of  inflexibility  to  respond  to  user’s  changing 
requirements.  However,  the  role  of  the  commander’s 
direct  representative  has  proven  to  be  highly  useful 
in  obtaining  exceptions  to  doctrinal  constraints  to 
improve  overall  system  responsiveness. 

The  third  configuration  has  essentially  “private 
line"  characteristics.  This  is  the  familiar  dedicated 
communications  concept.  Its  advantages  are  obvious. 
It  provides  the  ultimate  in  foreshortened  “writer  to 
reader"  time.  It  can  be  engineered  to  be  available 
when  needed.  It  seldom  saturates. 

The  disadvantages  of  this  type  of  configuration 
are  also  well  known.  It  is  more  prodigal  of  resources 
than  either  of  the  other  two  types  and  is  usually  less 
survivable  in  comparison  with  integrated  switched 
networks.  More  importantly,  however,  from  the  view¬ 
point  of  both  line  and  staff  command  hierarchies, 
this  configuration  has  the  inherent  characteristic  of 
comparting  data  and  information  to  an  inacceptable 
degree,  and  may  bypass  subordinate  echelons  of  the 
command. 

At  the  outset,  it  was  mentioned  that  these  several 
configurations  also  reflect  the  viewpoints  of  the  com- 
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mands  served.  Without  belaboring  this  aspect  of 
military  communications/information  systems,  it  may 
be  observed  that  those  highly  structured  military 
organizations  which  operate  with  well  established 
policies  and  doctrines  find  the  “public  utility”  con¬ 
cept  acceptable.  Those  organizations  whose  missions 
include  a  capability  to  react  in  extremely  short  times 
favor  the  “private  line"  concept. 

There  appears  also  to  be  a  direct  relationship  be¬ 
tween  the  “depth  of  control,”  the  length  of  the  chain 
of  command,  and  the  type  of  communications/in¬ 
formation  system  configuration  employed.  In  shal¬ 
lower  organizational  hierarchies,  dedicated  systems 
appear  more  frequently. 

Further,  command  organizations  that  traditionally 
are  highly  structured  with  detailed  doctrine  appear 
to  be  completely  distrustful  of  the  reliability  of  data 
tlow  in  any  actual  operational  situation. 

Conversely,  less  systematic  organizations  seem  to 
place  a  high  degree  of  faith  in  system  reliability  for 
the  conduct  of  military  operations  in  the  face  of  the 
enemy. 

The  term  “automated  information  system”  has  been 
used  in  connection  with  specific  military  applications 
of  a  wide  range  of  capabilities.  Consequently,  there 
have  arisen  major  semantic  difficulties  reflecting  the 
differences  in  viewpoints  of  the  sponsoring  agency, 
the  system  designer,  the  system  implementer,  and  the 
operational  user. 

Some  military  automated  information  systems,  in¬ 
tended  to  support  an  operational  military  commander 
and  his  staff,  have  been  labeled  “command  and  control 
systems.”  One  definition  of  this  aggregate  of  capa¬ 
bilities  states:  “A  command  and  control  system 
is  ...  a  composite  of  equipment,  skills,  and  techniques 
which,  while  not  an  instrument  of  combat,  is  capable 
of  performing  the  clerly  defined  function  of  enabling 
a  commander  to  exercise  continuous  control  of  his 
forces  and  weapons  in  all  situations  by  providing 
him  with: 

•  the  information  needed  to  make  operational 
decisions,  and 

•  the  means  for  disseminating  these  decisions. 

A  complete  system  includes  all  subsystems,  related 

facilities,  equipment,  material,  services,  and  personnel 
required  for  operation  of  the  system,  so  that  it  can 
be  considered  a  self-sufficient  unit  in  its  intended 
environment.” 

This  all-inclusive  view  of  a  command  and  control 
system  would  properly  encompass,  for  example,  an 
automated  weapons  control  subsystem,  although  its 
contribution  to  the  information  needs  or  decision¬ 
making  capability  of  the  commander,  or  its  demands 
upon  his  decision-making  capability,  may  be  relatively 


small.  Operating  as  essentially  closed  loops,  from 
target  to  weapon  back  to  target,  most  automated 
weapon  control  systems  are  characterized  by  integral 
sensing,  selection  of  an  alternative  in  conformance 
with  pre-established  doctrine  or  by  online,  subordi¬ 
nate  level  decision,  and  automatic  commands  to  the 
weapon  to  destroy  a  specific  target. 

Such  weapon  control  systems  normally  require  only 
a  simple  decision  (input)  by  the  commander  to  initi¬ 
ate  or  to  terminate  their  use.  Their  data  outputs  to 
the  commander  are  generally  restricted  to  periodically 
reporting  their  status,  targets  engaged,  resources 
remaining. 

For  purposes  of  this  discussion,  automated  mili¬ 
tary  command  and  control  systems  will  be  considered 
those  intended  to  satisfy  requirements  of  the  com¬ 
mander  and  his  staff  for  mechanical  and  electrical 
assistance  in  carrying  out  the  following  military 
functions: 

•  Sensing  significant  perturbations  in  the  situa¬ 
tion  or  environment. 

•  Storage,  retrieval,  transmission  of  data. 

•  Manipulation  and  analysis  of  data. 

•  Development  of  alternatives. 

•  Dissemination  of  decisions. 

Several  significant  presumptions  are  inherent  in 
so  describing  automated  military  command  and  con¬ 
trol  systems: 

•  It  presumes  a  universal  understanding,  and 
acceptance,  of  the  content  and  context  of  the 
command  function. 

•  It  presupposes  an  equally  universal  understand¬ 
ing  of  command’s  line  hierarchy. 

•  It  assumes  a  differentiation  between  data  and 
information. 

Technically,  its  design  includes  both  a  communica¬ 
tions  capability  and  a  data  processing  capability. 

Radical  changes  in  scope  and  methodology  are 
taking  place  in  modern  business  management.  Auto¬ 
mated  data  are  modifying  the  content  of  managerial 
jobs  at  all  levels,  the  roles  of  middle-  and  lower-level 
managers,  and  the  entire  structure  of  decision-making 
within  the  enterprise. 

To  the  extent  that  military  operational  command 
may  be  similar  to  business  management,  equally 
radical  changes  might  be  anticipated  in  military 
command  methods  and  organizations.  Improved  capa¬ 
bilities  of  communications  and  awareness  of  data 
processing  applications  in  business  have  prompted 
military  commanders  to  review  their  situations  and 
seek  to  obtain  an  automated  command  and  control 
system.  These  reviews  also  have  fairly  consistently 
seemed  to  indicate  the  desirability  of  centralizing 
the  data  — a  consequence  of  a  “total  system”  ap¬ 
proach,  popular  since  about  I960. 
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The  total  system  implies  that  all  data  are  collected, 
transmitted,  processed,  and  distributed  to  potential 
users. 

In  highly  structured  situations,  in  which  virtually 
all  contingencies  are  foreseen,  in  which  preplanning 
is  thorough  and  the  resulting  plans  and  doctrine  are 
established,  the  total  system  approach  can  be  ap¬ 
proximated. 

The  thermostatic  control  is  an  example  of  such  a 
total  system.  Temperature  and  humidity  ranges  can 
be  maintained  within  pre-established  limits;  even 
a  record  of  environmental  changes  and  system  re¬ 
sponses  could  be  obtained  very  simply,  from  which 
performance  analyses  and  logistic  support  require¬ 
ments  could  be  derived. 

However,  a  review  of  military  organizations  involv¬ 
ing  extensive  communications  and  data  manipulation 
reveals  that  the  total  system  approach  must  be  used 
with  caution  when  one  considers  the  nature  of  military 
activities. 

The  total  system  concept  presupposes  acquisition 
of  data  direct  from  source  by  means  of  communica¬ 
tions  capability  integral  to  the  automated  informa¬ 
tion  system.  Data  so  acquired  may  or  may  not  be 
available  to  intermediate  echelons  of  the  military 
organization. 

Assuming  the  technically  optimum  case,  wherein 
data  from  a  source  are  simultaneously  available  to  all 
command  echelons,  unilateral  decisions  by  higher 
headquarters  based  on  these  data  will  deprive  the 
higher  command  of  the  judgment  of  intermediate 
echelons. 

In  the  case  wherein  the  data  sources  differ  for  indi¬ 
vidual  commands  or  for  functional  subsystems  serving 
a  single  command,  problems  of  ambiguity  arise. 

In  the  event  that  the  upward  flow  of  data  follows 
the  command  hierarchy  and  is  forwarded  only  after 
evaluation  in  the  context  of  the  forwarding  head¬ 
quarters,  the  ensuing  delays  are  usually  considered 
unacceptable  by  the  sponsoring  agency,  the  system’s 
designer,  and  the  system’s  implementers.  The  opera¬ 
tional  users’  viewpoint  is  usually  ambiguous. 

What  seems  to  be  neglected  is  that,  in  the  military, 
the  command  hierarchy  is  and  remains  a  multilevel 
pyramidal  arrangement  of  headquarters,  each  differ¬ 
ing  from  the  others.  On  the  same  echelon,  the  differ¬ 
ence  between  commands  lies  in  specialization  and 
functional  cognizance.  Between  echelons  of  com¬ 
mand,  the  differences  in  contexts  stem  from  percep¬ 
tion  and  scope  of  interest.  This  hierarchy  is  a  result 
of  pragmatic  empiricism,  in  the  presence  of  constraints 
of  communications  and  the  limits  of  an  individual’s 
ability  to  cope  with  issues  in  the  time  available. 


The  constraints  of  communications  and  individual 
perception  which  are,  in  part,  responsible  for  the 
hierarchy  also  contribute  to  the  demand  for  filtered, 
evaluated  data  (i.e.,  information)  between  successive 
levels  of  command. 

At  higher  echelons  of  military  command,  there  is 
relatively  less  problem-solving,  in  which  the  situation 
is  postulated,  requirements  are  evident,  and  the  de¬ 
cision,  tactical.  In  this  context,  problem-solving  im¬ 
plies  a  single  optimum  solution.  The  situation  is 
given,  and  requirements  are  evident.  The  optimum 
solution  is  the  most  economical  adaptation  of  avail¬ 
able  resources. 

Rather,  higher  command  echelons  are  faced  with 
the  necessity  of  determining  the  situation,  even  of 
changing  the  situation;  of  determining  resources, 
either  what  they  are,  or  should  be;  of  developing  al¬ 
ternative  courses  of  action  or  options,  which  are  sel¬ 
dom  black  or  white  and  may  include  doing  nothing. 
Decisions  are  strategic  in  nature.  Further,  there  is  a 
continuing  requirement  to  identify  missing  informa¬ 
tion,  which  in  turn  generates  a  requirement  for  judg¬ 
ment,  and  for  determining  the  degree  of  precision 
allowable  or  acceptable  in  the  decision. 

At  higher  command  echelons  it  is  a  rare  situation 
indeed  in  which  all  needed  information  is  available 
and  there  is  one  optimum  solution.  In  fact,  wherever 
analysis  of  the  situation  leads  to  this  comforting  con¬ 
clusion,  one  may  reasonably  suspect  the  solution  of 
being  little  more  than  a  plausible  argument  for  a  pre¬ 
conceived  idea. 

Intermediate  and  lower  echelons  of  command  can 
contribute  directly  to  definition  and  analysis  of  the 
higher  headquarters  problem,  to  the  development  of 
alternatives,  and  to  the  conversion  of  any  decision 
to  effective  action;  and  can  make  these  contributions 
with  due  consideration  of  the  risks,  economy  of  ef¬ 
fort,  timing  and  limitation  of  resources  as  understood 
by  these  subordinate  headquarters. 

It  is  not  apparent  to  me  that  any  automated  com¬ 
mand  and  control  system  designed  to  date  has  ade¬ 
quately  taken  into  account  the  characteristics  of  the 
military  environment  described  above. 

For  example,  one  impact  of  automated  information 
upon  the  military  command  hierarchies  might  be  il¬ 
lustrated  from  considering  the  consequence  of  the 
introduction  of  radar  in  the  U.  S.  Navy  early  in 
World  War  II.  Initially  viewed  as  a  substantial  im¬ 
provement  in  the  sensory  range  of  the  naval  lookout, 
its  potential  contribution  as  a  source  of  data  and  in¬ 
formation  to  the  command  echelon  was  not  recog¬ 
nized  by  the  designers  nor  the  military  at  the  outset. 

Because  radar  provided  not  only  detection,  but 
target  location,  its  use  was  promptly  pre-empted  for 
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target  location,  its  use  was  promptly  pre-empted  for 
dedicated  application  to  weapons  control  problems, 
often  to  the  detriment  of  its  improved  “lookout” 
or  initial  detection  function. 

Later,  when  radar’s  potential  for  providing  data 
for  a  multiplicity  of  uses  was  recognized,  two  major 
changes,  one  technical  and  one  organizational, 
occurred: 

•  Technically,  the  basic  concept  for  presentation 
of  data  had  to  be  changed  from  theA-scope 
to  the  plan  position  indicator,  the  PPI. 

•  Organizationally,  a  new  group  of  operator- 
technicians  emerged  at  the  staff  level  further 
to  process  the  data  now  available  in  light  of 
on-going  operations,  not  only  of  their  own  com¬ 
mand  but  of  adjacent  commands  as  well. 

One  might  have  hoped  that  the  trend  taken  in  estab¬ 
lishing  this  new  group  would  have  enjoyed  universal 
acceptance  of  the  obvious  benefits  it  could  demon¬ 
strate. 

Actually,  the  notion  that  data  should  be  available 
to  several  potential  users  as  a  basis  for  each  user’s 
contribution  to  overall  command  analysis  and  de¬ 
cision  was  far  from  universally  accepted  by  function¬ 
ally  oriented  staff  elements.  On  the  contrary,  today, 
25  years  after  the  introduction  of  radar  one  finds 
military  organizational  arrangements  still  geared  to 
unilateral  use  of  dedicated  sensing  systems,  to  the 
exclusion  of  other  potential  users  in  the  command 
who  may  either  duplicate  the  data  collection  and 
processing  capability,  or  do  without  it. 

Sonar  is  another  example  of  a  space  sensing  system 
viewed  primarily  as  a  dedicated  data  input  for  ASW 
weapons  control,  rather  than  as  a  potential  source  of 
data  for  rapid  collation  with  other  undersea  space 
sensing  systems  and  intelligence  sources,  in  both 
strategic  and  tactical  applications. 

A  more  recent,  and  probably  more  widely  recog¬ 
nized  example  of  the  impact  of  an  automated  military 
command  and  control  system  on  the  military  com¬ 
mand  hierarchy  is  SAGE. 

Disregarding  service  orientated  opinions  as  to  the 
merits  of  its  technical  design,  SAGE  has  had  a  pro¬ 
found  impact  upon  military  organization  and  mission. 
It  provided  a  massive  detection,  data  collection, 
data  processing,  and  data  communications  capability 
which  made  technically  feasible  a  highly  centralized 
command  organization.  In  turn,  this  new  command 
undertook  the  detailed  management,  on  a  near  real 
time  basis,  of  the  air  defense  mission  for  the  entire 
North  American  Continent  — a  span  of  control  of 
military  operations  never  previously  envisioned. 


Current  policies  for  acquisition  of  automated 
command  and  control  systems  to  support  military 
command  seem  to  stipulate: 

•  Evolutionary  improvement  of  capabilities, 
exploiting  technological  advances  as  they  occur 
and  operating  experience  gained  in  copying  with 
changing  situations. 

•  Flexibility  in  system  performance  to  anticipate 
possible  future  changes  in  the  operational  en¬ 
vironment. 

•  Compatibility  with  associated  automated  in¬ 
formation  systems  of  coequal,  subordinate  or 
superior  echelons  in  the  command  hierarchy. 

Implementation  of  these  policies  can  lead  to  the 
procurement  of  off-the-shelf  ADP  equipment  and 
software,  the  latter  subsequently  modified  as  neces¬ 
sary  in  an  effort  to  tailor  the  off-the-shelf  ADP  capa¬ 
bilities  to  evolving  requirements. 

One  obvious  consequence  of  the  introduction  of 
this  type  of  automation  into  a  military  environment 
is  its  impact  upon  organization:  An  entirely  new 
organizational  entity  is  required  (but  seldom  ac¬ 
quired)  to  provide  for  data  management  and  computer 
program  implementation  and  maintenance.  The  Na¬ 
tional  Military  Command  System  Support  Center, 
operated  by  the  Defense  Communications  Agency  in 
support  of  the  National  Military  Command  System, 
and  the  Naval  Command  System  Support  Activity 
operated  by  the  Navy,  are  two  examples  of  these 
relatively  new  organizational  adjuncts  to  command. 

Understandably,  such  supporting  units  seek  to 
achieve  an  in-house  capability  to  convert  information 
requirements  of  command  into  usable  computer  pro¬ 
grams  to  produce  useful  information. 

However,  the  ADP  operator-technician-program¬ 
mer  has  had  no  significant  decision-making  experi¬ 
ence  upon  which  to  base  his  interpretation  of  user 
demands.  Moreover,  the  user  group  is  inherently 
distrustful  of  any  data  system  with  which  they,  as 
individuals,  are  unlikely  to  have  had  any  extensive 
personal  experience. 

One  hears  criticism  of  the  technical  ADP  staff 
as  lacking  “operational  experience.”  This  criticism 
should  be  interpreted  more  precisely  as  this  ADP 
staff  lacks  “decision-making  experience”  at  the 
command  level  beintf  served.  Issues  of  risks,  stakes, 
and  alternatives  are  implicit  in  this  view. 

On  the  other  hand,  senior  decision-makers  at  the 
command  level  are  equally  unlikely  to  be  conversant 
with  capabilities  and  limitations  of  ADP,  either 
now  or  in  the  immediate  future. 

Further,  provision  of  mass  data  systems  on  an 
incremental  basis,  using  hardware  and  software  de¬ 
signed  primarily  for  industrial  or  business  applica- 
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tions,  appears  to  presuppose  similarity  between  the 
techniques  of  business  management  and  of  military 
operational  command. 

As  pointed  out  previously,  there  are  marked  dif¬ 
ferences  in  the  operational  contexts  at  successive 
levels  in  the  classical  military  command  hierarchy. 

Decision  can  range  from  problem-solving  solutions 
at  the  lower  echelons,  to  determination  and  election 
of  alternative  courses  of  action  at  the  higher  levels. 

Techniques  to  support  this  spectrum  of  activity 
will  seldom  be  similar  throughout,  and  data  require¬ 
ments  will  not  be  similar  at  each  level  of  command. 

Nevertheless,  automated  command  and  control 
systems  using  off-the-shelf  A  DP  are  being  introduced 
to  support  higher  headquarters.  These  systems  gener¬ 
ate  requirements  for  large  amounts  of  data,  stored  at 
a  central  data  repository,  and  efficiency  of  processing 
data  demands  that  these  data  be  acquired  in  standard¬ 
ized  formats. 

But  differences  in  information  requirements  at 
intervening  headquarters  — not  differences  in  ADP 
equipment  and  programs  — inevitably  demand  com¬ 
promises  in  reporting  formats  and  in  data  content, 
or,  alternatively,  parallel  reporting  systems,  or  inde¬ 
pendent  levies  for  information  upon  subordinate  or 
collateral  commands. 

Next,  an  effort  is  made  to  obtain  "all”  data  for 
storage  at  the  central  higher  echelon  repository,  just 
in  case  these  data  may  be  needed.  The  rapid  pro¬ 
cessing  of  vast  amounts  of  data  leads  higher  echelons 
of  command  to  believe  they  possess  adequate  in¬ 
formation  upon  which  more  detailed  decisions  can 
be  based.  The  command  organization  at  the  higher 
echelons  then  tries  to  participate  actively  not  only  in 
strategic  decision  but  tactical  problem-solving  as 
well. 

To  the  extent  that  the  higher  command  actually 
makes  more  tactical  decisions,  there  is  a  diminution 
of  the  role  of  intermediate  headquarters,  its  experience 
in  making  decisions  is  reduced,  and,  in  the  event 
portions  of  the  centralized  data  acquisition,  pro¬ 
cessing,  or  dissemination  functions  fail,  catastrophic 
loss  of  overall  military  capability,  rather  than  a  more 
graceful  degration  of  performance,  can  well  result. 

Historically,  military  plans,  doctrine  and  proce¬ 
dures  evolved  to  minimize  the  otherwise  mandatory 
reliance  upon  communications,  to  ensure  standard 
patterns  of  behavior  and  timely  response  by  subordi¬ 
nates  in  given  situations,  and  to  provide  a  residual, 
though  reduced,  capability  in  the  event  of  partial  loss 
of  the  command  hierarchy. 

Improved  communications  and  data  processing 
capabilities  seem  to  have  considerably  lessened  the 


need  for  highly  structured  doctrine;  in  fact,  as  long 
as  it  survives,  the  capability  to  obtain  more  and  more 
data  rapidly  to  determine  a  situation,  rather  than  to 
postulate  it,  further  reduces  the  traditional  role  doc¬ 
trine  plays  in  military  operations,  or  substitutes  for 
it  a  more  complex  and  more  rapidly  changing  modus 
operandi. 

The  concomitant  ability  to  communicate  tactical 
decisions  directly  to  the  action  unit  in  turn  generates 
the  necessity  for  a  higher  degree  of  flexibility  of  re¬ 
sponse  by  the  action  unit,  which  in  turn  generates  a 
requirement  for  rapid  status  and  response  reporting 
back  to  the  command  level  initiating  the  tactical 
decision,  which  in  turn  generates  a  requirement  for 
more  communications  and  more  data  processing,  ad 
infinitum. 

SUMMARY 

To  summarize,  it  is  my  opinion  that  military  organi¬ 
zation  is  a  result  of  pragmatic  development.  The 
principal  tool  of  command  is  information.  The  com¬ 
mander  has  resources,  such  as  weapons,  personnel, 
and  vehicles  at  his  disposal,  but  he  cannot  manipu¬ 
late  these  resources  effectively  without  this  tool. 

The  command  echelons  have  been  structured,  with 
staff,  line,  and  doctrinal  delegates,  to  obtain  data, 
to  manipulate  data  to  derive  information  and  arrive 
at  alternative  courses  of  action,  and  to  communi¬ 
cate  decisions,  within  the  constraints  of  available 
or  nonavailable  communications. 

Increased  capabilities  in  communications  and  in 
data  processing  have  vastly  improved  the  ability  of 
commanders  to  acquire  data,  to  have  it  manipulated, 
and  to  transmit  derived  decisions. 

The  automated  information  systems  now  technically 
feasible  cover  a  wide  spectrum  of  capabilities,  rang¬ 
ing  from  individual  sensor  systems  generating  data 
for  one  or  a  variety  of  users,  through  weapons  control 
systems  of  varying  degrees  of  automatic  response, 
to  mass  data  systems  collecting  and  collating  large 
amounts  of  detailed  data.  However,  this  spectrum  of 
automated  information  systems  has  widely  divergent 
characteristics  in  terms  of  military  utility. 

The  sensor  system’s  utility,  with  its  design  based 
on  relatively  simple  concepts  and  known  doctrine, 
is  limited  only  by  engineering  ingenuity  and  ability 
to  control  the  physical  environment.  At  the  other 
end  of  the  spectrum  of  application,  mass  data  systems 
have  vast  technical  capability  but  virtually  no  doc¬ 
trinal  basis  for  design  and  employment,  other  than 
conventional  business  management  relationships. 

Automated  information  systems,  other  than  com¬ 
mand  and  control  systems,  employed  in  the  role  of 
sensor  systems  to  generate  data,  may  have  either  a 
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relatively  little  or  a  major  impact  upon  military  or¬ 
ganization  and  mission.  It  is  difficult  for  either  the 
designer  or  the  user  to  anticipate  the  magnitude  of 
this  impact. 

Higher  order  automated  information  systems,  such 
as  weapons  control  systems,  even  though  operational 
concepts  stem  from  established  doctrine  or  can  be 
stated  concisely  and  any  required  communications 
are  integral,  nearly  always  have  major  impacts  on 
both  military  organization  and  military  mission  at  the 
middle  echelon  level.  The  designer  and  user  jointly 
can  usually  predict  the  magnitude  of  this  impact. 

Automated  information  systems  intended  to  sup¬ 
port  higher  echelons  of  command  always  have  a 
profound,  continuing,  predictable,  but  seldom  recog¬ 
nized  impact  upon  both  military  organization  and 
military  missions. 


New  organizations  with  the  mission  of  ADP 
management  and  program  maintenance  are  required 
within  the  command.  The  tactical  decision-making 
capability  apparently  provided  to  higher  headquarters 
by  these  systems  tends  to  obviate  the  necessity  for 
making  decisions,  and  thus  the  learning  experience, 
at  intermediate  echelons  in  the  operational  chain  of 
command. 

Finally,  although  the  primary  purpose  of  informa¬ 
tion  systems  at  higher  headquarters  is  to  assist  in 
the  strategic  decision-making  process  — that  is,  de¬ 
termination  of  real-world  situations  and  development 
of  alternatives  — automated  data  processing  capa¬ 
bilities  continue  to  be  woefully  deficient  in  identify¬ 
ing  missing  information,  exercising  judgment,  or  de¬ 
termining  the  degree  of  precision  acceptable,  in  the 
light  of  risks  and  stakes  involved  and  the  resources 
available. 
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INTRODUCTION 

Te  purpose  of  this  paper  is  to  stimulate  thinking  on 
the  relationship  between  the  information  handling 
funetions  associated  with  command  and  control  and 
changing  trends  in  military  management.  The  intent 
is  neither  to  discuss  the  functions  of  command  and 
control  nor  to  compare  the  values  of  centralized  and 
decentralized  management  and  information  systems. 
The  paper  attempts  to  illustrate  how  the  character 
of  organizations  and  related  management  functions 
are  influenced  by  information  processing  technology. 
Thus  it  discusses  how  management  practices  may 
change  as  better  information  handling  capabilities 
are  provided,  and  how,  in  turn,  these  changes  further 
influence  the  character  of  the  related  information 
handling  functions. 

In  the  interest  of  brevity  and  in  an  effort  to  confine 
the  scope  of  the  paper,  many  military  responsibilities 
and  functions  are  described  only  in  the  context  of 
this  paper.  Accordingly,  these  descriptions  are  not 
wholly  complete  and  may  be  erroneous  if  applied 
in  a  different  context. 

Part  one 

Present-day  military  planning  is  characterized  by 
a  broad  spectrum  of  military  conflict,  widely  scat- 
ered  areas  of  operations,  the  involvement  of  many 
agencies,  and  the  concept  of  rapid  mobility.  For  ex¬ 
ample,  the  European  and  Pacific  threatres  of  opera¬ 
tion  each  have  large  forces  and  resources  continu¬ 
ously  assigned  to  the  area.  They  can  be  augmented 
as  required  with  forces  and  resources  normally  po- 

♦The  thoughts  expressed  in  this  paper  are  those  of  the  authors,  and 
do  not  represent  the  position  of  the  Department  of  Defense  or  any 
Agency  thereof. 


sitioned  in  the  United  States.  On  the  other  hand, 
there  are  potential  conflict  areas  where  little  or  no 
forces  and  resources  arc  in  place,  and  for  which  pre¬ 
planned  packages  must  be  organized  and  deployed 
as  the  need  arises. 

The  concept  of  mobility  packages,  and  the  general 
capability  available  for  moving  large  force  pack¬ 
ages,  permits  a  rapid  response  to  many  potential 
levels  and  areas  of  conflict  without  indefinitely  com¬ 
mitting  forces  to  each  area.  While  this  permits  world¬ 
wide  coverage  with  less  “inventory,”  it  greatly  in¬ 
creases  the  requirement  for  centralized  planning  and 
coordination. 

Thus  the  problem  becomes  one  of  competing  re¬ 
quire  ents  for  finite  resources.  No  one  CINC* 
or  Service  or  Single  Manager  can  unilaterally  make 
the  decisions  while  significantly  affect  several  other 
Defense  organizations.  Questions,  such  as  how  much 
of  the  MACf  fleet  should  be  held  in  reserve,  whether 
to  mobilize  CRAFff  aircraft,  what  the  impact  would 
be  of  borrowing  weapons  from  one  theatre  to  use  in 
another,  or  the  price  to  be  paid  for  shortening  closure 
times,  can  often  only  be  answered  at  the  high,  com¬ 
mon  level  of  the  National  Command  Authority 
(NCA).  It  is  now  recognized  that  the  NCA  requires 
the  capability  for  rapidly  assessing  feasible  alterna¬ 
tives.  This  obviously  has  profound  implications  with 
respect  to  information  processing  and  organizational 
responsibilities. 

There  arc  clearly  many  functions  which  a  CINC, 
Military  Department,  Single  Manager,  or  Component 

*Commander-in-Chief 

t Military  Airlift  Command  (formerly  Military  Air  transport 
Service- MATS). 
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can  do  best.  However,  resource  planning  on  a  world¬ 
wide  basis,  particularly  when  reaction  time  is  at  a 
premium,  must  often  be  done  at  a  higher  command 
level.  In  this  respect,  it  is  important  to  note  that 
most  of  the  details  with  which  each  organization  is 
normally  concerned  will  still  be  addressed  by  that 
organization.  It  is  the  policy  making,  establishing  of 
priorities,  and  the  rapid  determination  of  the  role  each 
agency  will  play  in  an  unplanned  environment  that 
must  be  the  function  of  the  highest  level. 

At  a  minimum,  the  information  processing  capa¬ 
bility  supporting  the  NCA  level  must  be  able  to  ag¬ 
gregate,  correlate,  ana'yze,  and  evaluate  that  data  and 
information  provided  by  all  of  the  potentially  affected 
agencies.  The  question  of  level  of  detail  of  unpro¬ 
cessed  data  to  be  forwarded  to  the  NCA  is  of  critical 
importance,  since  the  end  result  of  the  information 
processing  at  the  NCA  level  is  intended  to  be  the  as¬ 
signment  of  responsibilities  to  each  of  the  major  or¬ 
ganizations  involved. 

It  is  not  at  all  clear,  however,  that  this  will  be  where 
NCA  involvement  will  stop.  Given  the  data  process¬ 
ing  and  communications  to  acquire  and  process  vast 
quantities  of  data,  the  NCA  can,  if  they  wish,  ef¬ 
fectively  monitor  and  exercise  considerable  control 
over  the  manner  in  which  each  organization  carries 
out  many  of  its  responsibilities.  In  the  past,  experi¬ 
ence  has  shown  that  the  best  balance  is  achieved 
by  retaining  only  centralized  policy  making,  overall 
broad  planning,  and  monitoring  of  operations  at  the 
NCA  level,  and  by  decentralized  control  of  opera¬ 
tions  at  lower  levels.  Each  supporting  organization 
must  therefore  have  its  own  information  processing 
capability  for  detailed  planning,  and  must  also  con¬ 
tribute  to  the  NCA  broad  planning  role.  This  means, 
for  example,  that  the  NCA  can  determine  the  feasi¬ 
bility  and  capability  of  MAC  support  of  a  mission, 
or  to  assess  the  impact  of  utilization  of  aircraft  on 
closure  time,  but  leaves  to  MAC  the  problem  of  de¬ 
termining  and  controlling  flow  schedules  and  cycling 
of  aircraft  in  coordination  with  other  affected  agencies. 

This  concept  is  based  on  the  premise  that  the  single 
most  important  problem  in  both  long-range  planning 
and  crisis  planning  is  the  coordination  of  all  affected 
agencies.  Under  noncrisis  conditions,  the  NCA  could 
afford  the  time  required  for  numerous  studies  to  be 
made  by  competent  organizations,  each  feeding  in¬ 
formation  to  the  other,  with  many  interactions  be¬ 
fore  a  final  result  is  attained.  Under  crisis  conditions, 
or  when  a  multitude  of  plans  must  be  feasibility-tested 
in  a  short  time,  the  classic  approach  to  planning  may 
need  to  be  drastically  curtailed. 

The  most  valuable  benefit  of  centralized  planning 
is  that  a  great  many  alternatives  can  be  considered. 


i.e.,  a  thorough  sensitivity  analysis  can  be  performed, 
in  less  time  than  was  often  required  to  examine  one 
untested  course  of  action.  As  noted  previously,  how¬ 
ever,  there  is  an  attendant  danger.  Since  communica¬ 
tions  and  data  processing  technology  impose  no 
practical  limitation  on  how  much  raw  data  can  be  ac¬ 
quired  and  stored,  it  is  possible  for  the  NCA  to  have 
in  the  centralized  data  base  all  of  the  data  held  by 
all  other  organizations.  (Often  the  highly  detailed 
raw  data  is  required  to  insure  that  the  aggregations 
of  the  data  is  accurate.)  But  this  level  of  detail  also 
permits  the  NCA  to  “second  guess”  the  judgment 
and  actions  of  operational  organizations.  Movement 
schedules  can  be  developed  as  easily  at  the  NCA 
level  as  at  any  other  level.  The  problem  arises  there¬ 
fore  in  the  manner  in  which  parameters  are  applied 
and  results  interpreted. 

Part  two 

Present  day  technology,  i.e.,  the  present  state-of- 
the-art,  imposes  very  little  constraint  on  the  determi¬ 
nation  of  centralization  vs  decentralization,  i.e., 
what  functions  are  carried  out  by  what  echelons. 
This  is  entirely  a  matter  of  the  personalities  involved, 
and  the  willingness  to  implement  a  system  which  will 
permit  the  degree  of  command  and/or  control  inter¬ 
relationships  desired.  In  other  words,  any  desired 
organizational  structure  can  be  supported  from  an 
information  processing  point  of  view;  however,  the 
cost/effectiveness  issue  may  impose  realistic  con¬ 
straints.  Management  now  has  at  its  disposal  an  im¬ 
pressive  array  of  tools;  the  question  is,  how  does 
management  choose  to  use  what  is  available? 

The  previous  point  leads  to  a  consideration  of  w  hat 
is  perhaps  the  most  significant  issue.  From  the  broad 
view-point,  what  is  the  impact  of  information  pro¬ 
cessing  potential  on  organization  and  mission?  There 
are  many  facets  to  this  question.  The  question  we 
really  want  to  ask  is,  what  does  information  pro¬ 
cessing  capability  enable  us  to  do  at  this  time  that 
we  either  could  not  do  at  all  before,  or  could  do  only 
with  great  difficulty,  and  so  what  effect  does  this  have 
on  the  way  we  organize  and  the  functions  we  assign? 

First,  there  is  the  classic  case  of  using  information 
processing  to  automate  an  existing  manual  system. 
This  is  to  say  that  nothing  is  really  changed  except 
that  some  people  are  relieved  of  tedious  tasks,  or 
some  tasks  are  accomplished  faster.  This  is  a  very 
important  application,  and  the  first  and  most  important 
task  of  system  analysis  is  to  determine  whether  or 
not  simply  automating  the  manual  processing  is  what 
is  required.  For  example:  Optical  character  readers 
can  enable  direct  input  of  data  to  a  computer,  speed¬ 
ing  up  the  process  and  requiring  less  personnel;  mes¬ 
sages  can  be  logged  automatically;  operational  data 
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can  be  extracted  from  files  much  faster  than  from  a 
multitude  of  messages  and  reports,  etc.  Little  if  any¬ 
thing  changes  in  the  fundamental  way  that  the  or¬ 
ganization  works. 

By  the  same  token,  it  is  possible  for  an  information 
processing  system  to  enable  a  whole  new  spectrum 
of  functions  to  be  performed  simply  because  vast 
quantities  of  data  can  be  processed  at  a  central  lo¬ 
cation,  very  quickly,  and  minute  details  considered 
along  with  all  implications,  thus  bypassing  many 
echelons  previously  needed  for  the  same  information 
processing.  Now  there  can  be  a  requirement  for  ad¬ 
ditional  staff  elements  to  perform  analyses  and  help 
formulate  policy,  functions  which  were  previously 
accomplished  at  a  lower  echelon  or  by  a  special  group 
established  to  provide  these  services.  There  are  many 
examples  within  the  Services  and  industry  of  this 
trend. 

It  is  interesting  to  note  that,  in  the  case  where  both 
information  processing  and  decision-making  are  cen¬ 
tralized,  there  is  not  necessarily  a  modification  to  the 
existing  organization.  One  of  the  most  significant 
and  obvious  manifestations  of  this  is  the  manner  in 
which  the  National  Command  Authority,  specifically 
the  President,  Secretary  of  Defense,  and  the  JCS 
interface  more  or  less  directly  with  subordinate 
commands  of  the  CINCs.  Part  of  this  is  due  to  com¬ 
munications,  which  enable  almost  real  time  reporting 
of  events.  Part  is  also  due  to  the  ability  to  store  and 
process  vast  quantities  of  data  w  hich  previously  would 
have  been  aggregated  by  the  CINC  and  forwarded 
to  the  NCA  as  summary  reports. 

Although  the  CINCs  remain  in  the  direct  line  of 
command,  data  flows  directly  from  their  subordinate 
field  commands  to  Washington.  This  data  is  simul¬ 
taneously  furnished  to  the  CINCs  and  intermediate 
levels  so  that  they  may  process  it  for  their  own  pur¬ 
pose,  know  as  much  about  the  situation  as  the  au¬ 
thorities  in  Washington,  and  be  prepared  to  discuss 
it  with  them.  The  availability  of  all  such  information 
in  the  Washington  area  tends  to  generate  questions 
and  discussions  on  matters  concerning  many  echelons 
of  command.  This  in  turn  tends  to  encourage  direct 
discussions  between  authorities  in  Washington  and 
the  field  forces  involved.  Intermediate  echelons 
should,  of  course,  participate  in  such  discussions. 
This  way  of  doing  business  does  not  necessarily 
mean  that  all  decisions  are  made  in  Washington, 
but  the  pressure  for  rapid  and  complete  answers  to 
questions  and  the  implied  requirement  for  solutions 
to  problems  is  far  greater  than  before. 

Two  possible  approaches  to  data  reporting  and 
processing  might  be  considered:  One  is  to  have  all 
data,  however  detailed,  sent  directly  to  the  highest 


echelon.  In  this  case,  not  only  can  the  detailed  data 
be  aggregated  to  provide  summary  reports  appropri¬ 
ate  at  the  highest  level,  but  the  data  can  be  used  by 
persons  not  familiar  enough  with  current  operational 
problems  to  draw  valid  inferences.  For  example, 
planning  factors  modified  by  recent  operational  ex¬ 
perience  may  not  be  available  to  accompany  the  raw 
data.  There  is  no  question  that  the  same  detailed 
data  can  be  used  at  all  levels.  The  question  is  what 
processing  capability  should  be  available  to  the  highest 
echelon? 

The  second  approach  is  to  have  the  detailed  data 
held  in  data  bases  at  lower  levels,  not  to  be  tapped 
as  raw  data,  but  to  be  processed  or  partially  processed 
at  that  or  an  intermediate  level  for  forwarding  to  the 
higher  level.  Thus,  when  certain  reports  were  re¬ 
quired,  the  reports  themselves,  or  summarized  data, 
would  be  available  to  the  higher  level  on  demand. 
This  would  avoid  having  the  data  available  with  the 
attendant  danger  of  using  it  incorrectly.  This  also 
enables  the  lower  echelon  to  have  at  its  disposal 
data  which  can  provide  the  basis  for  answering  more 
detailed  questions  which  may  arise  without  being 
pre-empted. 

Part  three 

The  organization  of  command  and  control  functions 
has  a  significant  influence  on  the  organization  of 
the  associated  information  systems  and  vice  versa. 
It  appears  that  there  are  at  least  five  general  con¬ 
cepts  which  might  evolve: 

•  The  organization  remains  the  same  and  the 
functions  remain  the  same;  information  pro¬ 
cessing  merely  enables  more  efficient  carrying 
out  of  existing  functions. 

•  The  organization  remains  the  same,  but  function¬ 
al  responsibilities  change,  e.g.t  a  lower  echelon 
no  longer  has  the  same  degree  of  local  control 
that  it  once  had,  some  of  this  control  having 
been  assumed  at  higher  echelons. 

•  One  or  more  lower  echelons  are  eliminated  or 
cut  out  with  respect  to  being  in  the  chain  of 
command  upward,  although  they  still  remain  in 
the  downward  chain. 

•  The  lower  echelons  can  have  semi-autonomous 
local  control,  providing  information  properly 
aggregated  to  higher  echelons,  resulting  in 
much  closer  coordination  without  giving  up  all 
prerogatives. 

•  The  information  processing  can  be  centralized 
with  an  attendant  increase  in  local  control.  The 
only  significant  organizational  change  in  this 
instance  is  the  technical  support  at  the  higher 
echelon. 
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by  Robert  E.  Harshbarger 

Defense  Communications  Agency 
Washington,  D  C. 


Most  discussions  of  information  systems  consider  the 
subject  in  its  continuum  from  source  data  gathering 
through  the  various  steps  of  communications  to  a 
central  location,  automatic  screening  and  manipula¬ 
tion  of  the  data,  display  or  presentation  to  a  user,  the 
user  analysis  and  evaluation  process  and  finally  the 
decision  and  action  response  process.  To  the  implement- 
er  this  represents  a  series  of  separate  though  closely 
interdependent  implementation  processes  demanding 
differing  techniques,  technologies,  disciplines  and  sup¬ 
porting  organizations.  To  concentrate  on  implementa¬ 
tion  areas  having  some  commonality  of  technology  and 
organization,  this  discussion  will  address  only  that 
segment  of  organization  concerned  with  implementing 
the  data  screening  and  manipulation  process  and  the 
process  of  providing  the  resulting  data  or  information 
to  the  user;  in  essence,  the  concern  of  automatic  data 
handling  from  the  time  data  is  received  at  a  central 
location  through  its  application  to  a  user  problem. 
Organizational  elements  concerned  with  data  gathering 
and  reporting,  with  communications  systems  to  support 
this  reporting  and  with  decision  and  action  response 
are  not  specifically  addressed  although  the  remarks 
presented  are  believed  to  be  generally  applicable  to 
these  areas  of  implementing  activity. 

Any  discussion  of  information  systems  and  their 
impact  on  organization  and  mission  or  on  any  other 
element  of  management  or  technology  upon  which 
they  impinge  must  be  preceded  by  some  descriptive 
argument  reconciling  the  ambiguities  of  the  term  itself. 
This  term,  information  system,  like  many  others  in  the 
current  technical  jargon  bears  clear  definition  only  in 
the  mind  of  the  individual  applying  the  term.  The 
vision  may  focus  at  any  point  in  the  spectrum  from 
the  most  rudimentary  manual  methods  of  collecting 
data  through  the  varying  degrees  of  complexity  as 
found  in  large  manual  systems  for  reporting  or  sum¬ 
marizing  data,  computer  assisted  systems  for  providing 
a  data  bank,  automated  systems  permitting  inquiry 
of  the  data  bank,  analog  devices  for  converting  directly 
observed  data  to  useful  information,  complex  systems 


of  multiple  users  interacting  independently  in  apparent 
real  time  with  a  large  automated  central  data  bank, 
and  on  to  the  vastly  complex  and  interconnected  sys¬ 
tems  of  men  and  machines  as  may  be  beheld  only  by 
the  visionary  divorced  from  practicality.  To  select  from 
this  span  of  diverse  concepts  common  threads  impact¬ 
ing  on  organization  and  mission  is  not  possible  due  to 
the  diversity  of  the  supporting  organizations  themselves. 
Rather,  in  order  to  focus  the  discussion  to  points  ger¬ 
mane  to  this  Congress,  it  is  necessary  to  delimit  the 
meaning  of  the  term  to  some  specific  concepts  and 
capabilities  recognized  in  current  computer  automated 
systems  or  anticipated  in  the  following  generation  of 
systems. 

The  term,  information  system,  as  subsequently  ref¬ 
erenced  will  include  computer  systems  containing  an 
alphanumeric  data  bank  accessible  in  apparent  real 
time  by  each  of  a  number  of  users,  including  automatic 
response  to  critical  parameter  thresholds,  to  obtain 
read-out  of  data  or  to  derive  information  from  such 
data  and  providing  the  means  for  rapid  display  of  the 
data  or  information  so  derived.  While  this  somewhat 
restricted  definition  is  not  necessarily  descriptive  of 
current  national  level  command  control  data  handling 
systems  with  which  I  am  closely  associated  and  toward 
which  I  will  generally  direct  my  remarks,  and  does 
not  reflect  any  specifically  approved  planning  toward 
such  goals,  it  is  representative  of  that  minimum  sys¬ 
tem  I  believe  necessary  to  satisfy  current  demands  at 
the  national  level.  Indeed,  in  the  absence  of  well  de¬ 
fined  goals,  an  assumed  definition  is  necessary  to  the 
implementing  organization  if  chaos  is  not  to  reign 
supreme.  The  implementing  activity  is  one  close  to 
reality,  solving  today’s  problems  today,  making  deci¬ 
sions  on  methods  and  techniques  applicable  today  but 
which  influence  tomorrow’s  decisions  through  the 
impact  of  economic  investment  and  continuity  of  appli¬ 
cations,  and  providing  system  growth  and  transition 
to  meet  daily  changing  requirements.  Orderly  and  eco¬ 
nomically  sound  growth  demands  well  defined  goals 
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whether  specified  or  assumed.  The  remarks  which  fol¬ 
low  will  not  propound  philosophy  nor  argue  justifica¬ 
tion  of  systems’  being.  Rather,  they  stress  the  need 
for  clear  goals,  whatever  the  justification,  and  the  im¬ 
pact  of  the  existence  or  absence  of  such  goals  on  to¬ 
day’s  implementing  organization  and  activities. 

The  dynamic  growth  of  information  technology  dur¬ 
ing  the  past  10  years  has  given  rise  to  a  number  of 
automated  military  information  systems  of  varying  com¬ 
plexity  utilized  in  storage  and  manipulation  of  alpha¬ 
numeric  data.  Existence  of  these  information  systems, 
though  generally  they  may  be  better  described  as  data 
banks,  has  brought  recognition  of  the  wide  variety  and 
large  volumes  of  data  that  may  be  efficiently  stored 
and  manipulated.  This  facility  for  storage  and  manipu¬ 
lation  has  in  turn  broadened  the  demand  for  more  and 
more  diverse  and  detailed  data  accumulation  at  vari¬ 
ous  command  levels.  The  past  three  years  have  seen 
unprecedented  advances  in  communications  to  support 
this  demand  and  most  notably  have  witnessed  the  evo¬ 
lution  of  broad  and  well  disciplined  reporting  systems 
oriented  toward  utilization  of  computer  methods  in 
data  transmission  and  data  handling.  These  mecha¬ 
nisms  allow,  and  indeed  today  are  accomplishing, 
the  accumulation  of  great  volumes  of  data  at  many 
levels  of  command.  Unfortunately  our  alaerity  in  pro¬ 
ducing  the  means  for  translating  these  large  data  vol¬ 
umes  into  information  assimilable  by  the  interested 
user  has  not  matched  our  progress  in  providing  the 
data  itself.  I  somewhat  sanguinely  presume  that  this 
will  be  the  next  revolution  in  the  information  systems 
evolution. 

Throughout  the  history  of  systems  relying  on  analog 
methods  for  obtaining  source  data  the  conversion  to 
usable  information  has  been  an  accepted  part  of  the 
system.  This  derives  both  from  the  generally  unintel¬ 
ligible  nature  of  the  source  data  to  the  ultimate  user 
and  from  the  great  volume  of  individual  data  inputs. 
In  the  typical  radar  or  sonar  application  the  automatic 
conversion  of  source  data  to  a  few  blips  on  a  scope 
depicting  range  and  location  of  detected  objects  is 
only  an  initial  step  which  partially  reduces  the  vast 
volume  of  source  data.  Useful  information  derives  only 
after  extensive  distillation  of  many  such  observations 
into  some  operationally  meaningful  statement  of  number 
of  hostiles  penetrating,  their  relative  location,  their 
course  or  closing  rate,  expected  time  of  closure,  and  sim¬ 
ilar  factual  details  on  which  decisions  can  be  made. 
Such  information,  which  in  no  way  conveys  the  vast  vol¬ 
ume  of  source  data  from  which  it  was  distilled,  is  com¬ 
monly  accepted  as  system  output.  Similarly,  the  often 
referenced  Semi-Automatic  Ground  Environment  Sys¬ 
tem  initially  grew  from  these  same  precepts  of  the  ana¬ 
log  system.  It  was  early  recognized  that  higher  and  high¬ 


er  levels  of  aggregate  information  were  necessary  to 
rapid-response  decisions  required  in  the  air  defense  mis¬ 
sion.  Moreover  this  aggregation  has  been  carried  to  the 
integration  of  information  from  widely  dispersed  cen¬ 
ters;  information  aggregated  to  a  level  of  context  com¬ 
pletely  removed  from  the  source  data.  Here  again  infor¬ 
mation  reflecting  in  no  way  the  vast  conglomerate  of 
source  data  is  accepted  by  the  operational  user. 

Current  alphanumeric  information  systems  have  be¬ 
come  quite  analogous  in  complexity  to  these  analog 
systems;  data  volumes  reporting  facts  and  detail  of 
facts  have  become  far  too  numerous  for  assimilation  by 
the  individual  user;  data  coding  makes  direct  reading 
almost  unintelligible;  complexities  of  the  interdepen¬ 
dence  and  interaction  of  these  detailed  facts  preclude 
intuitive  judgments  and  decisions;  and  distillations  of 
these  data  volumes  into  more  comprehensive  levels  of 
information  are  necessary  if  decision  responsiveness  is 
to  be  effective.  The  simple  data  bank  or  inventory  con¬ 
trol  approach  which  accepts  source  data  and  subse¬ 
quently,  only  on  demand,  spews  forth  minutia  or  sum¬ 
marized  data  sorted  in  many  ways  will  provide  an 
initial  level  of  volume  reduction  comparable  to  that 
evidenced  by  the  blips  on  the  radar  scope.  Automatic 
distillation  to  much  higher  levels  of  information  eon- 
tent  are  not  possible  in  the  data  bank  approach  since 
neither  the  hardware  nor  software  associated  with  this 
approach  adequately  support  the  voluminous  arithmet¬ 
ical  and  logical  calculations  necessary  to  the  distilla¬ 
tion.  Even  the  simplest  extension  of  information  level 
to  the  sensing  of  parameter  thresholds  eannot  be  effec¬ 
tively  incorporated  in  the  data  bank  environment. 
Some  such  sensings  that  are  at  best  limited  include 
indications  that  an  event  has  or  has  not  occurred, 
parameter  value  has  exceeded  some  specified  bounda¬ 
ry,  rate  of  parameter  value  change  has  exceeded  some 
specified  norm  or  that  experienced  in  a  previous 
time  period,  or  that  the  fraction  of  remaining  resource 
capability  has  fallen  below  some  specified  critical  level. 
Yet  such  minimum  extension  is  necessary  to  direct 
the  user’s  attention  to  areas  of  critical  change  or  occur¬ 
rence  demanding  further  investigation  or  action  re¬ 
sponse;  else  he  must  make  frequent  and  time-consum¬ 
ing  inquiry  into  the  status  of  a  large  number  of  indi¬ 
vidual  parameter  conditions  or  must  peruse  volumi¬ 
nous  batch  processed  printouts  or  paper  listings  to  sift 
out  these  more  critical  changes  as  they  oecur.  Both 
time  for  such  random  inquiry  and  the  volume  of  data 
detail  to  be  perused  preclude  effective  decision  response 
as  data  inputs  continue  to  grow  in  size,  detail  and  time¬ 
liness.  Further  extension  to  information  distillates  con¬ 
veying  the  impact  on  operations  of  the  interaction  of 
multiple  parameters  involved  in  operations  planning, 
status  of  forces  and  resources,  plan  execution  and  fol- 
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lowing,  and  transportation  and  logistics  support  ean  be 
accomplished  only  by  a  comprehensive  information 
system  as  opposed  to  a  data  bank. 

Distinct  from  the  personal  reasoning  above,  the  im¬ 
plementing  organization,  as  an  organization,  is  indif¬ 
ferent  to  why  a  specific  concept  is  accepted.  It  needs 
only  a  generally  clear  definition  of  the  near  term  and 
longer  range  goals  toward  which  the  system  is  aspiring. 
These  goals  must  distinguish  between  data  banks  and 
information  systems  or  levels  between,  must  be  general¬ 
ly  accepted  by  management  and  operating  officials 
at  all  levels,  and  must  be  achievable  in  a  reasonable  and 
generally  accepted  time  frame.  The  impact  on  the  im¬ 
plementing  activity  of  the  system  definition  implied 
by  such  goals  is  very  direct  in  both  mission  and  organi¬ 
zation.  As  regards  the  data  bank  definition  on  the  one 
hand,  mission  encompasses  little  more  than  providing 
a  storehouse  for  data  and  distributing  the  data  in 
various  forms  on  demand  to  multiple  users;  the  more 
comprehensive  information  system,  on  the  other  hand, 
extends  mission  scope  to  include  development  of  the 
necessary  tools  to  permit  automatic  response  as  speci¬ 
fied  conditions  arc  met,  to  allow  multiple  users  to 
interact  on-line  with  the  data  base,  to  permit  complex 
queries  and  analyses  involving  multiple  parameters, 
and  to  provide  rapid  display  and  distribution  of  signif¬ 
icant  results.  These  obvious  differences  in  mission 
dictate  widely  varying  organization  structures,  demand 
quite  diverse  technologies,  require  quite  different  aca¬ 
demic  disciplines  and  experience  backgrounds,  and  im¬ 
ply  completely  contrary  views  of  user  function.  The 
implementing  organization,  not  being  clairvoyant,  re¬ 
quires  adequate  mission  definition  if  its  efforts  are  to 
be  efficiently  directed.  The  alternative  is  to  hedge 
against  the  entire  span  of  possibilities  thus  ensuring 
that  a  large  proportion  of  its  efforts  are  misdirected  if 
not  totally  lost. 

H.  D.  Benington  noted  in  1964  in  a  somewhat 
broader  context  that  “  .  .  .  almost  everyone  empha¬ 
sizes  evolutionary  system  design,  not  fully  appreciating, 
however,  the  eventual  impact  that  this  new  approach 
will  have  on  the  role  of  technologists,  the  goals  of 
our  research,  the  development  of  tools,  and  the  orga¬ 
nization  of  the  command.”  At  the  level  of  the  imple- 
menter,  evidence  docs  not  indicate  that  two  years  have 
added  significantly  to  the  appreciation  of  the  impact 
of  this  evolutionary  approach  other  than  to  change 
the  verb  tense  of  Mr.  Benington’s  statement  from  “will 
have”  to  “is  having.” 

Distinction  between  the  evolution  of  hardware  sys¬ 
tems,  software  tools  and  applications  programs  is  not 
generally  recognized.  To  the  implementing  ageney  this 
is  reflected  in  a  general  slowdown  or  lack  of  progress 
in  applied  research  while  information  systems  as  refer¬ 


enced  herein  require  a  vigorous  research  program, 
the  basic  role  of  the  technologist  is  virtually  ignored 
as  initial  increments  of  the  evolution  almost  totally 
absorb  available  resources,  development  of  information 
systems  tools  specifically  for  the  evolutionary  needs 
of  the  military  environment  receives  only  minimal 
interest  (other  than  in  communications)  defeating  the 
very  concept  of  evolution,  and  organization  at  best 
becomes  unstable  due  to  the  lack  of  appreciation  of 
the  above.  Information  systems  coupled  with  an  evolu¬ 
tionary  approach  require  that  the  designer  and  imple- 
menter  provide  a  series  of  tools  developed  outside 
the  user  environment  to  ensure  a  flexible  system  within 
which  the  user  can,  in  his  own  environment,  develop 
programs  and  procedures  applicable  to  specific  prob¬ 
lems.  These  tools  incorporate  individually  eomplex 
control  programs,  programming  languages,  inquiry 
languages,  console  devices,  data  transmission  systems, 
mathematical  simulation  and  utility  systems,  and  data 
management  and  manipulation  systems  into  a  single 
complex  in  which  they  effectively  and  efficiently  inter¬ 
act  in  a  manner  easily  accessible  to  the  user  and  the 
applications  programmer.  Information  systems  there¬ 
fore  require  that  the  implementing  organization  include 
an  element  devoted  to  the  applied  research,  develop¬ 
ment  and  implementation  of  these  tools.  The  com¬ 
plexity  of  the  technology  required  for  this  develop¬ 
ment  demands  expert  technical  and  logical  talent  ex¬ 
perienced  in  the  disciplines  of  complex  computer  sys¬ 
tems  and  strong  technical  management  to  ensure  com¬ 
patibility  of  the  many  complex  elements.  This  pre¬ 
sumes  that  the  evolution  is  directed  toward  the  goal 
of  an  information  system  rather  than  toward  a  data 
bank  which  can  be  provided  by  well-known  technology 
with  minimal  developmental  requirements.  Since  or¬ 
ganizational  elements  and  organizational  competence 
are  not  created,  but  are  built  over  a  period  of  months 
or  years,  distinct  goals  toward  which  to  build  are 
essential.  Based  on  a  common  understanding  applied 
research  can  be  initiated,  acquisition  and  retention  of 
technological  competence  can  be  accomplished  based 
on  known  discipline  requirements,  tools  serving  the 
continuing  system  evolution  can  be  developed,  and  an 
organizational  structure  responsive  to  the  technological 
requirements  and  to  the  needs  of  the  user  and  manage¬ 
ment  can  be  established. 

Information  systems  bring  to  the  implementer  the 
need  for  strong  centralized  control  in  both  the  imple¬ 
mentation  and  operation  of  the  system.  The  many 
components  of  the  central  computer  control  system 
arc  so  complexly  interrelated  that  a  single  manager  is 
necessary  during  implementation  to  ensure  the  efficient 
functioning  of  these  parts  as  a  total  system.  Since  such 
a  system  serves  many  functional  users  a  single  manager 
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must  recognize  the  needs  of  these  many  users  to  en¬ 
sure  that  system  scope  is  sufficiently  broad,  both  func¬ 
tionally  and  in  detail,  to  meet  these  needs.  He  must 
also  control  any  changes  or  additions  to  the  system 
since  even  a  minor  change  in  one  component  may 
impact  on  many  other  components.  In  operational  use 
strong  control  is  necessary  to  reflect  priorities,  deter¬ 
mine  saturation  points,  detect  and  correct  system 
errors,  and  most  importantly  to  ensure  that  all  users 
are  aware  of  all  available  standard  algorithms  and 
routines;  the  latter  to  prevent  multiple  duplication  of 
programming  and  often  inefficient  programming  of 
frequently  used  routines  which  are  the  two  greatest 
contributors  to  inefficiency  in  large  information  sys¬ 
tems. 

The  implementing  activity  must  provide  the  organi¬ 
zational  element  and  disciplines  to  assist  the  user  in 
the  application  of  these  automatic  tools  to  his  specific 
information  needs.  The  problems  of  the  military  user 
have  become  vastly  more  complex  during  the  recent 
years  that  have  embraced  the  policy  of  controlled  re¬ 
sponse  and  have  at  the  same  time  seen  the  availability 
of  more  and  more  detailed  data  relating  forces,  plans 
and  actions  at  all  command  levels.  This  has  required 
a  more  complex  logical  technology  in  evaluation  of  the 
military  environment.  Man  has  of  necessity  learned 
to  cope  with  this  increased  complexity  and  contin¬ 
uously  becomes  more  technically  sophisticated  in  order 
to  do  so.  The  military  user  is  indeed  the  expert  in  his 
own  environment;  understanding  the  problems;  able 
to  define  what  information  is  needed  for  their  solution; 
capable  of  defining  the  general  methods,  procedures 
and  problem  logic  necessary  for  developing  applica¬ 
tions  programs.  The  implementer  need  only  to  guide 
in  the  use  of  the  automatic  tools  and  assist  in  aetual 
programming  where  required.  Yet,  too  often,  the  user 
is  coerced  into  spending  valuable  time  in  learning 
the  details  of  computer  programming,  usually  at  a 
machine  language  level  remote  from  any  application 
he  will  ever  encounter,  in  an  attempt  to  unravel  the 
mysteries  surrounding  his  information  system.  In  order 
to  provide  the  user  with  the  time  to  properly  do  the 
job  for  which  he  exists  rather  than  become  an  expert 
in  noncontributory  peripheral  areas,  the  implementer 
must  provide,  as  part  of  the  user  assistance,  a  mech¬ 
anism  for  keeping  the  user  informed  of  system  capabil¬ 
ities  and  limitations  and  the  procedures  for  applying 
the  automatic  tools  available  to  him.  These  should  be 
explainable  and  applicable  in  technical  English,  math¬ 
ematics,  and  logic  without  reference  to  the  cryptic 
language  of  the  programmer  and  program  systems. 
Presentation  must  be  in  a  manner  which  instills  confi¬ 
dence  in  the  system,  denoting  simplicity  and  ease  of 
application.  To  provide  less  will  only  compound  the 


already  complex  problems  of  the  user  and  divert  him 
from  his  primary  functions.  Here  the  technical  com¬ 
petence  of  the  implementer  and  of  the  scientific  com¬ 
munity  in  providing  such  tools  is  being  tested,  not 
that  of  the  user.  Provided  adequate  tools  and  succinct 
guidance  in  their  application  the  user  will  advance  at 
a  rate  dictated  by  his  own  requirements. 

The  phenomenal  growth  and  advancing  technology 
of  information  systems  also  brings  to  the  implementer 
the  traditional  problems  of  an  expanding  organization. 
Frequent  change  in  organization  structure  to  provide 
response  to  new  functional  areas,  success  and  failure 
in  the  investigation  and  research  into  new  technologies, 
justification  of  increased  resources,  acquisition  and 
retention  of  competent  personnel,  training  in  applica¬ 
tion  of  new  technologies,  and  confronting  squarely 
the  decision  required  in  the  face  of  uncertainty  are 
all  internal  problems  with  which  the  implementer  can 
cope  so  long  as  his  goals  are  well  defined.  In  the 
absence  of  such  goals  each  of  these  areas  become,  in 
themselves,  a  time  consuming  and  wasteful  process 
bent  on  outguessing  the  future. 

While  all  of  the  areas  of  concern  heretofore  men¬ 
tioned  impact  directly  on  the  implementing  organiza¬ 
tion,  none  are  insurmountable  in  the  presence  of  clear¬ 
ly  established  goals.  Presuming  their  existence,  one 
area  of  significance  remains,  that  is,  the  need  for 
conveying  to  the  basic  research  and  design  activities 
the  requisite  changes  and  advancements  in  the  over¬ 
all  system.  The  implementer,  being  continously  in 
close  contact  with  the  users  in  their  applications,  is 
the  first  to  recognize  the  need  for  major  changes  or 
additions  to  the  then  existing  system  components.  In 
an  evolutionary  environment  these  requirements  for 
change  generally  appear  in  small  increments  and  often 
are  incorporated  as  minor  improvements  or  as  part 
of  system  maintenance.  They  derive  from  the  obser¬ 
vations  of  several  organizational  elements  and  emerge 
as  basic  design  changes  only  after  experience  shows 
the  need  for  some  general  improvement  in  overall 
capability;  frequently  in  the  form  of  a  need  to  make 
more  efficient  some  processes  already  in  being.  A 
specific  element  of  the  implementing  organization  with 
the  mission  requirement  of  uncovering  and  defining 
such  needs  is  necessary  if  continuing  liaison  between 
the  design  organization  and  subelements  of  the  imple¬ 
menting  organization  is  to  be  maintained.  The  need 
must  be  transmitted  in  some  formal  fashion  from  either 
the  user  or  implementer  to  the  design  activity.  This 
separate  organization  activity  will  inevitably  remain 
one  step  removed  from  the  system  and  the  user  thus 
diluting  its  effectiveness.  The  manner  in  which  the 
need  for  basic  design  changes  derive,  the  neeessary 
formality  of  transmitting  recognized  design  needs  to 
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elements  even  further  removed  from  the  system,  and 
the  necessity  of  separate  organizational  elements  main¬ 
taining  detailed  knowledge  of  systems  programs  and 
procedures  make  this  process  at  least  cumbersome 
and  time  consuming  if  not  inefficient  with  inadequate 
response.  From  the  view  of  the  implemcntcr  the  mis¬ 
sion  areas  of  information  system  design  and  imple¬ 
mentation  are  not  separable  where  the  concept  of 
evolutionary  system  development  prevails. 

The  current  challenge  to  the  scientific  community 
and  to  the  implementing  activity,  in  the  continuing 
evolution  of  information  systems,  lies  in  the  full  reali¬ 
zation  of  the  inundating  volume  of  data  rapidly  be¬ 
coming  available  to  the  user  and  the  concomitant 


demand  for  the  tools  which  will  allow  the  user  to 
automatically  translate  this  volume  into  higher  levels 
of  information  assimilable  at  the  decision  level.  Fur¬ 
ther,  though  the  tools  themselves  may  be  complex,  it 
must  be  recognized  that  the  view  presented  to  the  user 
should  be  one  of  simplicity  and  ease  of  use,  one  which 
instills  confidence  in  the  utility  of  the  tools  rather  than 
compounding  the  problems  and  impairing  the  technical 
efficiency  of  the  user.  Equally  important  is  the  chal¬ 
lenge  to  management  at  all  levels  of  recognizing  these 
current  demands,  generally  defining  the  longer  term 
goals,  and  of  providing  the  policy  guidance  and  re¬ 
sources  through  which  the  tools  can  be  developed 
and  their  applications  exploited. 
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INTRODUCTION 

The  purpose  of  this  paper  is  to  establish  the  basic 
characteristics  of  a  comprehensive  and  flexible  capa¬ 
bility  for  strategic  mobility  analysis.  This  objective 
arises  from  the  recent  establishment  within  the 
Organization  of  the  Joint  Chiefs  of  Staff  of  the  Office 
of  the  Special  Assistant  for  Strategic  Mobility 
(SASM). 

SASM  is  charged  explicitly  with  a  wide  range  of 
missions.  Governing  the  full  spectrum  of  strategic 
mobility,  SASM  is  charged  implicitly  with  executing 
his  strategic  mobility  functions  with  the  best  analysis 
technology  now  available. 

This  analysis  technology  has  three  major  com¬ 
ponents.  The  primary  impetus  is  that  of  systems 
analysis,  the  generic  term  for  rigorous  analysis  of 
difficult  problems.  The  second  is  the  technology  of 
computers,  required  by  the  complexity  of  strategic 
mobility  problems.  7'hc  third  technology  is  the 
accumulated  knowledge  of  transport  systems  analy¬ 
sis,  together  with  the  associated  mathematical  and 
computer  models. 

In  response  to  this  implicit  charge,  SASM  has 
under  way  a  major  effort  to  develop  a  broad  analysis 
capability  for  strategic  mobility.  Development  of 
this  capability  requires  maximum  use  of  the  systems 
analysis  approach,  of  computer  support,  and  of 
transportation  system  models.  The  purpose  of  this 
paper  is  to  discuss  the  basic  characteristics  of  this 
analysis  capability. 

We  begin  with  a  discussion  of  the  strategic  mobility 
problem,  in  order  to  clarify  the  substantive  area  of 
concern.  Next,  we  give  a  relatively  cursory  discussion 
of  issues  in  developing  models  for  use  in  strategic 
mobility.  Third,  we  discuss  systems  analysis  as  a 
framework  and  indicate  the  limitations  of  this  frame¬ 
work  and  its  implications  for  analysis  procedures. 
Fourth,  we  discuss  briefly  the  kind  of  computer 
environment  which  can  be  provided,  and  the  impli¬ 
cations  this  environment  has  for  design  of  an  analysis 


capability.  Finally,  we  summarize  the  implications 
of  the  preceding  discussions  by  identifying  the  basic 
characteristics  of  a  strategic  mobility  analysis  capa¬ 
bility. 

The  arguments  of  this  paper  are  by  no  means  de¬ 
finitive  and  final,  but  are  presented  with  the  objective 
of  stimulating  discussion  in  the  technical  communities 
concerned  with  strategic  mobility  analysis. 

/.  The  Strategic  Mobility  Problem 

A.  The  triangle  of  resources,  requirements 
and  criteria. 

The  strategic  mobility  problem  can  be  represented 
by  a  triangle,  whose  three  corners  are;  strategic 
movement  resources,  strategic  movement  require¬ 
ments;  and  criteria  for  selection  of  strategic  mobility 
plans  (Figure  1 ). 


Requirements 


Figure  I  —The. strategic  mobility  triangle 
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Strategic  movement  resources  are  all  the  trans¬ 
portation  resources  available  for  support  of  world¬ 
wide  mobility,  including  military  owned,  commercial 
for-hire,  and  non-military  vehicles  and  networks  avail¬ 
able  under  alternative  conditions.  These  are  enumer¬ 
ated  in  Figure  2. 

A.  Vehicles 

Aircraft 

MAC  nucleus 

Commercial,  voluntarily  available 
Civil  Reserve  Air  Fleet 
Non-U. S.  owned 
Ships 

MSTS  nucleus 

U.S.  commercial,  voluntarily  available 
Flags  of  Convenience 
Other  foreign-flag  commercial,  voluntarily 
available 
Reserve  fleets 
Other  available 
CONUS  and  Theaters 

Rail  cars —  box,  flat,  heavy  duty  flat,  POL, 
passenger 

Trucks  — commercial  for-hire,  private  carrier; 
military —  organic,  non-organic,  other  gov¬ 
ernment-owned 

Buses  — commercial,  private  carrier,  military, 
other 

Inland  waterways 
Special  movements 
LSD’s,  LCM’s 

Aircraft  carriers  and  other  helicopter 
transports 
Floating  cranes 
New  technologies 
Ground-effects  machines 
Containers 

B.  Networks 

Installations 

CONUS  origins  — home  stations,  depots 
Theater  destinations  — depots,  staging  areas 
Terminals 

Water  terminals  (POE’s  and  POD’s)  — 
constructed  and  LOTS  (logistics-over-the- 
shore) 

Air  terminals  (POE’s  and  POD’s) 

Enroute  air  terminals 
Enroute  sea  terminals 
Links 

CONUS  — rail,  highway,  inland  water,  coast¬ 
al 

Inter-theater  — air,  sea 

Theater— rail,  highway,  off-the  road,  inland 
water,  coastal 

Figure  2  — Strategic  movement  resources 


Strategic  movement  requirements  are  all  move 
ments  through  the  worldwide  transportation  system 
which  are  in  support  of  a  strategic  plan  or  which 
impact  on  the  ability  of  the  transportation  resources 
to  support  that  plan.  These  include  the  forces  being 
deployed,  including  personnel  fillers  and  replace¬ 
ments  and  resupply,  and  reverse  flows  from  the 
theaters,  as  well  as  civilian  and  non-deployment 
flows  within  the  the  theater,  within  the  Continental 
U.S.  (CONUS),  or  between  the  CONUS  and  the 
theaters  (such  as  existing  channel  traffic  and  house¬ 
hold  goods  and  dependents  movements).  These  are 
summarized  in  Figure  3. 

Units 

Personnel 

Equipment 

Accompanying  supply 
Personnel 
Fillers 

Replacements 
Resupply 
Reverse  Flows 

Civilian  dependents 
POW’s 

Medical  evacuees 
Prepositioning 
Equipment 
Supply 

Reinforcements  and  redeployments  worldwide 
Special  movement  requirements 
POL 

Helicopters 
A1 D  cargoes 
Channel  flows 

Support  of  military  forces 
Support  of  civilian  economy 
Figure  3  —  Strategic  movement  requirements 
The  third  element  of  the  triangle,  criteria  for  se¬ 
lection  of  a  mobility  plan,  includes  consideration  of 
the  movement  times  and  costs  (fixed  and  variable) 
associated  with  alternative  strategic  mobility  plans, 
and  considerations  of  effectiveness  of  plans  as  re¬ 
flected  in  the  effectiveness  of  the  forces  delivered, 
and  vulnerability  and  reliability.  These  are  further 
detailed  in  Figure  4. 

B.  Problem  types 

The  significance  of  this  triangle  is  that  it  focuses 
on  the  variety  of  problem  types  which  can  be  formu¬ 
lated  for  strategic  mobility  analysis.  Any  given  prob¬ 
lem  can  be  characterized  in  terms  of  the  three  vertices 
of  the  triangle,  by  indicating  which  vertices  are  speci¬ 
fied  in  the  problem  statement,  and  which  are  to  be 
found  by  solution  or  analysis  of  the  problem  (Figure 
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5).  Various  combinations  are  shown  below,  with  ex¬ 
amples: 

Time  (Arrival  at  destination) 

Completion  of  total  deployment 
Specified  phases:  by  force  packages,  or  by  mode 
of  delivery 

Actual  departure,  relative  to  ready  date 
Actual  arrival,  relative  to  required  dates 
Costs 

Capital  costs 

Operating  costs  — fixed  variable 
Effectiveness  of  forces  delivered 
Rate  of  force  buildup 
Unit  integrity 

Matchup  of  cargo  and  passengers 
In-transit  time  and  travel  conditions 
Time  left  available  for  training 
Flexibility 

Nature  and  location  of  constraints  on  flow 
Actual  utilization  of  movement  resources  as 
compared  to  potential 
Vulnerability 

Convoy  size  and  speed 
Dispersion  of  vehicles  and  units  over  time 
Dispersion  of  vehicles  and  units  over  space 
Essentiality  of  specific  links  or  terminals 
Figure  4 -Strategic  mobility  criteria 

(1)  Requirements  and  resources  are  specified,  find 
performance  with  respect  to  the  criteria: 

a.  What  time  will  it  take  to  deliver  the  require¬ 
ments  with  the  specified  resources? 

b.  What  is  the  cost  of  using  these  resources 
to  deliver  the  requirement? 

(2)  Requirements  and  criteria  arc  specified,  find 
the  resources  required: 

a.  Resources  required  to  meet  a  specified  de¬ 
ployment  completion  time? 

b.  Resources  required  for  minimum  cost  de¬ 
ployment  which  meets  the  required  time  and 
effectiveness? 

(3)  Resources  and  criteria  specified,  find  the  re¬ 
quirements  which  can  be  delivered: 

a.  Quantity  of  personnel  and  equipment  which 
can  be  delivered  in  a  given  time? 

b.  Within  specified  cost  limits? 

All  three  problem  types  arise. 

This  triangle  is  offered  as  a  conceptual  aid  only, 
for  it  does  submerge  many  subtle  aspects.  For  ex¬ 
ample,  these  problem  types  do  not  follow  the  patterns 
above  completely: 

(1)  Requirements  specified  for  different  time 
periods,  find  minimum  cost  use  of  transporta¬ 
tion  resources  over  all  time  periods. 


(2)  All  transportation  resources  specified  except 
for  the  number  of  new  large  aircraft,  which  is 
to  be  found.  This  is  a  special  case  of  the  more 
general  problem  of  finding  the  transportation 
resources  given  the  requirements  and  the 
criteria;  even  with  the  restriction  on  transporta¬ 
tion  resources,  the  requirements  and  criteria 
must  still  be  specified  for  the  problem  to  have 
meaning. 


Resource* 


Figure  S  —  Problem  types 

C.  Routing  and  scheduling 

The  core  of  the  strategic  mobility  triangle  is  move¬ 
ment  routing  and  scheduling.  At  the  most  detailed 
level  of  strategic  mobility  analysis,  a  complete  de¬ 
tailed  movement  schedule  is  actually  established,  for 
every  movement  unit  being  deployed.  This  schedule 
traces  the  unit  from  initial  origin  through  port  of  em¬ 
barkation  (POE)  and  port  of  debarkation  (POD)  to 
theater  destination,  indicating  for  each  leg  the  trans¬ 
portation  mode,  route,  vehicle  or  vehicles  assigned 
(e.g.,  ship  name,  number  of  aircraft  by  type),  and  de¬ 
tailed  timing  information  for  arrivals  and  departures 
(Figure  6).  Such  a  schedule  is  the  result  of  routing 
and  scheduling  through  multiple  transport  modes. 

When  the  type  of  analysis  does  not  require  the  level 
of  detail  represented  by  the  complete  movement 
schedule,  “routing  and  scheduling”  arc  still  present. 
Even  for  the  most  general  capabilities  studies,  rout- 
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Figure  6  — Detailed  movement  schedule 
ing  and  scheduling  decisions  are  implicit;  any  state¬ 
ment  about  the  capability  of  a  particular  transporta¬ 
tion  facility  is  based  upon  an  explicit  or  implicit  as¬ 
sumption  about  routing  and  scheduling  within  the  sys¬ 
tem.  For  example,  an  air  channel  capacity  of  1000 
short  tons  per  day  is  based  upon  assumptions  about 
types  of  aircraft,  operating  policies  and  routing  and 
scheduling  of  aircraft  and  crews  over  routes  in  and 
out  of  the  channel  (cycling  to  maintenance,  etc.). 

As  shown  in  Figure  7,  this  has  significant  implica¬ 
tions  for  the  analysis  of  strategic  mobility  problems. 
If  routing  and  scheduling  is  not  done  explicitly  in 
the  analysis,  assumptions  about  routing  and  schedul¬ 
ing  and  their  impact  on  channel  and  other  capabilities 
must  be  verified.  This  argument  is  particularly  im¬ 
portant  in  understanding  the  relationships  among 
different  types  of  transportation  models. 

There  are  four  types  of  routing  and  scheduling 
decisions: 

(1)  Mode  selection  decisions.  These  decisions  de¬ 
pend  upon  such  factors  as  vehicles  available, 
time  available  for  the  move,  route  capabilities 
(highways,  rail  lines,  enroute  airfields,  etc.), 
and  cost  (dollars  and  strategic  value  of  re¬ 
sources). 

(2)  Route  selection  decisions.  These  decisions  can 
be  broken  into  two  parts: 

a.  Selection  of  POE’s  and  POD’s,  which  de¬ 
pends  upon  such  considerations  as  the  mode 
selected  for  inter-theater  lift,  port  capability. 


and  port  vulnerability. 

b.  Selection  of  routes  for  a  single  mode,  which 
depends  on  such  factors  as  POE’s  and 
POD’s  chosen,  availability  of  enroute  sup¬ 
port  for  vehicles  being  used,  and  flexibility 
(potential  for  enroute  diversion  around  con¬ 
gested  or  vulnerable  links). 

(3)  Vehicle  selection  decisions  (within  a  mode). 
These  decisions  depend  on  such  considerations 
as  the  number  of  each  type  vehicle  available 
at  the  time  and  place  required,  and  the  total 
lift  capacity  of  available  vehicles  as  compared 
to  total  lift  requirements. 

(4)  Timing  decisions.  Establishment  of  movement 
schedule  times  depends  on  the  routes  selected, 
speed  of  vehicles  selected,  anticipated  queues 
enroute,  and  whether  scheduling  is  done  from 
the  availability  date  forward  (availability  date 
plus  travel  time  equals  predicted  arrival  time 
at  destination),  or  from  required  delivery  date 
back  (required  delivery  date  minus  travel  time 
equals  required  time  of  departure  from  origin). 


Requirements 


Resources 

Criterio 

Figure  7  —  Routing  and  scheduling 

These  routing  and  scheduling  decisions  interact 
greatly.  Decisions  of  each  type  will  be  influenced  by 
decisions  of  other  types.  Hence,  the  sequence  in 
which  these  decisions  are  developed  will  strongly  in¬ 
fluence  the  movement  schedule  developed,  and  the 
evaluation  of  capability. 

To  the  maximum  extent  possible,  it  is  desirable  to 
consider  the  total  transportation  system  as  a  single 
system,  not  just  one  part.  Decisions  cannot  be  made 
for  any  mode  in  isolation  — for  example,  sealift  or  air¬ 
lift.  Strategic  mobility  analysis  must  deal  with  factors 
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throughout  the  system,  including  the  large  number  of 
origins  within  CONUS,  various  unit  readiness  dates, 
various  required  delivery  dates,  various  tradeoffs 
between  modes  and  within  modes,  between  routes, 
between  speed  of  deployment  and  readiness  status, 
etc.,  as  well  as  various  constraints  (POL  availability, 
port  throughput  capacity,  cargo  needing  specialized 
handling,  etc.). 

D.  Time  frame  distinctions 

A  typical  division  of  strategic  planning  problems 
is  based  upon  time  frames: 

(1)  Long-range  planning  — over  five  years. 

(2)  Mid-range  planning  — one  to  five  years. 

(3)  Contingency  planning  — within  the  current 
year. 

(4)  Operational  planning  and  current  operations  — 
right  now  to  one  to  three  months. 

The  triangle  of  resources  — requirements  — criteria 
is  valid  for  each  time  frame.  The  dominant  aspect  of 
the  problem  is  the  relationship  of  the  problem  to  the 
triangle.  For  example,  it  does  not  matter  whether  we 
arc  dealing  with  the  transportation  force  structure  of 
the  one-five  year  range  or  the  over-five  year  range; 
what  is  most  critical  for  the  mobility  analysis  is 
whether  this  is  a  problem  of  the  resources-given  or  the 
resources-to-be-found  type.  Time  frame  distinctions 
play  a  significant  but  lesser  role  in  determining  the 
analytical  techniques  required. 

Certain  general  patterns  do  emerge,  in  normal 
planning  practice,  relating  the  strategic  mobility 
triangle  to  time  frames.  The  following  types  of  stra¬ 
tegic  mobility  problems  are  typical: 

(1)  Operational  Planning.  Transportation  re¬ 
sources  and  movement  requirements  are  specified 
generally  in  detail;  the  problem  is  to  determine  the 
most  efficient  application  of  resources  to  require¬ 
ments.  The  criteria  of  efficiency  are  predominantly 
time  and  effectiveness  of  forces  delivered. 

(2)  Contingency  Planning .  Transportation  re¬ 
sources  arc  fixed.  Planner  wishes  to  explore  costs, 
time,  and  force  effectiveness  implications  of  the  most 
efficient  application  of  the  transportation  resources 
to  alternative  levels  and  mixes  of  movement  require¬ 
ments. 

(3)  Force  Structure  Planning.  Several  sets  of 
movement  requirements  are  given.  Each  set  of  re¬ 
quirements  corresponds  to  a  different  contingency  or 
set  of  contingencies.  For  each  set  of  movement  re¬ 
quirements,  the  least-cost  mix  of  transportation  re¬ 
sources  required  is  to  be  found  for  alternative  levels 
of  efficiency  (time  and  effectiveness  of  forces  de¬ 
livered).  A  desirable  force  structure  is  determined  by 
“averaging”  over  all  the  sets  of  requirements. 


E.  Analysis  functions 

All  strategic  mobility  functions  can  be  summarized 
under  the  major  functional  tasks  of  Force  structure 
planning.  Contingency  planning,  and  Current  opera¬ 
tions.  For  instance,  the  function  of  monitoring  re¬ 
search  and  development  in  strategic  mobility  is 
primarily  concerned  with  the  introduction  of  new 
vehicles  and  transportation  technologies  in  the  long- 
range  time  frame.  The  general  issue  is,  how  docs  the 
performance  of  the  transportation  system  change  with 
changes  in  vehicles  or  other  technologies.  This  ques¬ 
tion  may  sometimes  be  applicable  to  current  opera¬ 
tions  also.  Therefore,  in  all  time  frames  of  analysis 
there  is  a  general  option  open  to  the  planner  to  ex¬ 
plore  changes  in  vehicle  characteristics. 

This  discussion  of  the  strategic  mobility  problem 
leads  to  Figure  8.  This  figure  shows  how  the  major 
functional  tasks  in  strategic  mobility  lead  to  decision 
issues.  These  issues  can  be  related  to  options  open  to 
the  planners,  which  in  turn  lead  to  the  basic  analysis 
problem,  balancing  resources  against  requirements. 
The  product  of  analysis  is  information  upon  which 
to  base  decisions. 


Functional  Taaka 


Fore a  atructura 
planning 
Con t Infancy 
planning 

Currant  oparatlona 


Declalon 

laauea 


Option* 


Analyala 


Dec la Iona 


Declalon  Iaauaa 

R&D  on  new  vehicla  type  or  new 
tranaportetlon  technology? 

Conetruct  new  port,  airfield, 
highway? 

Improve  or  cloae  exlatlng  faellltlaa? 

Improve  acceaa  to  poat,  canpe  and 
atatlona? 

Procure  additional  vehlclee? 

Relocate  tranaport  forces? 

Collect  more  trenaportatlon 
Intelligence? 

Contingency  plan  feasible? 

.  ixlmum  uae  of  resources? 

A  itlelpated  bottlenecks  In 
current  operations? 

Alternative  aolutlona? 

Degree  of  mobilization  to  declare? 

Improve  readlneas  atatue  of  unit a? 

What  ehould  ba  prepositioned? 

Change  orlglne  of  units? 


Options 

Resource* 

Changes  In  networks  and  taminela  - 
locations,  characteristics 
Changes  In  vehicles  available  - 
number,  characteristics, 
availability  rtites 
Changes  In  operating  policies  - 
utilization  rates,  allocation  of 
vehicles 

Routing  and  scheduling  of  rssouross 
Use  of  civilian  tranaport  - 

mobilization  of  0. S.  capabilities, 
uae  or  theater  labor  and  trans¬ 
portation  faellltlaa 
Requlrerentc 

Forces  to  be  deployed  -  destinations, 
tins  required,  specific*  units,  origins 
Supply  -  raoupply  ratee,  atockage 
levels,  prepositioning 
Loading  -  combat  or  administrative 
loadings 


Figure  8  —  Functions  of  analysis 

F.  Conclusions 

There  is  no  single  type  of  strategic  mobility  prob¬ 
lem.  A  system  for  strategic  mobility  analysis  must 
accept  different  formulations  corresponding  to  dif¬ 
ferent  aspects  of  the  triangle  and  different  time 
frames.  Inputs  to  the  system  will  generally  consist 
of  resources  and/or  requirements  and/or  performance 
criteria.  Even  if  routing  and  scheduling  is  not  done 
explicitly,  all  factors  and  assumptions  used  must  re¬ 
flect  explicit  consideration  of  implicit  assumptions 
about  routing  and  scheduling. 
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II.  Modelling  Strategic  Transportation 

The  basic  concern  of  strategic  mobility  is  trans¬ 
portation;  the  core  of  the  strategic  mobility  problem 
is  routing  and  scheduling  of  transport  vehicles.  Tech¬ 
niques  and  models  developed  for  other  transportation 
problems  are  applicable  to  strategic  mobility. 


and  control  and  propulsion  systems  technologies, 
together  with  a  broad  set  of  operating  policies.) 

Application  to  strategic  mobility  analysis 
The  strategic  transportation  system  includes  all 
modes  of  transportation  utilized  by  the  military,  in- 


A.  Principles  of  transport  systems  analysis 

A  number  of  general  principles  for  the  analysis 
of  transport  systems  are  shown  in  Figure  9  (a  more 
concise  statement  of  these  principles  is  found  in 
Manheim,  1966  b).  These  principles  have  direct  ap¬ 
plication  to  the  problem  of  modelling  the  strategic 
transportation  system. 

1.  Transportation  systems=networks+ 
vehicles. 

11.  All  modes  of  transportation. 

III.  All  movements. 

IV.  From  initial  origins  to  final  destinations, 
through  all  modes. 

V.  Decision  variables:  short-run  to  long-run 

VI.  Transportation-related  decision  variables  — 
particularly  those  influencing  demand. 

VII.  Full  set  of  consequences. 

VI 11.  “Market”:  supply  and  demand  reach 

equilibrium  within  the  constraining  channels 
of  the  transportation  network. 

IX.  Comparative  analyses  must  maintain  the 
same  set  of  assumed  conditions. 

X.  Transportation  is  not  an  end  in  itself. 

figure  9  — Principles  of  transport  systems  analysis 

Principle  l 

Transportation  systems  are  composed  of  networks 
and  vehicles  moving  through  those  networks.  Net¬ 
works  consist  of  nodes  and  links  which  connect 
pairs  of  nodes.  Some  nodes  may  be  interchange 
points  between  links  of  the  same  mode;  other  nodes 
may  be  interchange  points  between  links  of  different 
modes,  commonly  called  terminals. 

Application  to  strategic  mobility  analysis 

In  strategic  mobility,  the  transportation  system  of 
interest  is,  potentially,  the  complete  world-wide 
transportation  network.  A  highly  abstract  model  of 
this  network  is  shown  in  Figure  10. 

The  implications  of  this  principle  for  modelling 
strategic  transportation  are  sufficiently  great  that  we 
reserve  discussion  for  Section  11I-B. 

Principle  ll 

All  modes  of  transportation  must  be  considered. 
(A  mode  is  a  particular  set  of  vehicle,  supporting  way, 
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Figure  10  — Strategic  mobility  factors 
eluding  military-owned  air,  sea,  highway,  rail,  and 
terminal  transportation  capabilities;  and  the  same 
capabilities  in  private  or  other  governmental  owner¬ 
ship  and  potentially  utilizable  by  the  military.  (See 
Figure  2.) 

For  strategic  mobility  analysis,  models  must  be 
developed  for  all  modes  of  transportation.  Further, 
models  must  be  able  to  represent  proposed  new  tech¬ 
nologies. 

Principle  HI 

All  movements  through  the  transportation  system 
must  be  considered. 

Application  to  strategic  mobility  analysis 

Analysis  must  consider  all  movement  require¬ 
ments  moving  through  the  world-wide  transportation 
system  at  the  same  time:  units  and  accompanying 
supply  being  deployed;  personnel  fillers  and  replace¬ 
ments;  resupply;  reinforcements  and  redeployments 
world-wide;  reverse  flows  from  the  theaters;  existing 
or  assumed  channel  flows  to  support  forces  already 
deployed;  and  flows  to  support  the  civilian  economics 
in  the  CONUS  and  in  the  theaters.  (See  Figure  3.) 

Strategic  mobility  models  must  be  able  to  simulate 
the  flows  of  all  of  these  movement  requirements,  con¬ 
sidering  the  special  characteristics  and  resource  needs 
of  each. 

Principle  /V 

Movements  must  be  considered  from  their  initial 
origin  to  final  destination,  through  all  modes. 

Application  to  strategic  mobility  analysis 

Models  of  the  strategic  transportation  system 


An  Overview  of  Strategic  Mobility  and  its  Implications  127 


must  consider  the  movements  of  forees  and  supplies 
from  initial  origins  (home  stations  or  depots);  through 
the  CONUS  transportation  system  to  the  air  and  sea 
ports  of  embarkation  (POE’s);  through  the  inter¬ 
theater  air  and  sealift  systems  to  the  theater  ports 
of  debarkation  (PODVs);  and  through  the  theater 
transportation  systems  to  destinations  (staging  areas 
or  theater  depots).  (See  Figure  6.)  It  is  not  sufficient 
simply  to  model  the  airlift  phase.  Models  must  also 
consider  movements  from  one  theater  to  another  or 
from  theaters  back  to  the  CONUS. 

Models  must  consider,  not  only  the  performance 
of  single  modes  in  isolation,  but  also  interactions  be¬ 
tween  modes,  at  terminals  and  other  interface  points. 
Models  are  required  which  specifically  address  the 
relationships  between  movements  into  and  out  of 
interface  points  where  movements  transfer  from  one 
mode  to  another. 

Principle  V 

The  set  of  transportation  decision  variables  should 
eover  the  full  scope  from  short-run  to  long-range 
options: 

a  long-range  investments,  such  as  changes  in 
the  Fixed  facilities  (networks  and  terminals), 
size  and  composition  of  available  vehiele  fleets, 
and  changes  in  transportation  technologies 
(vehicles,  supporting  ways,  ete.); 

b.  mid-range  options,  such  as  options  with  regard 
to  the  procurement,  distribution  and  allocation 
of  vehiele  resources,  and  network  operating 
policies  (routing  and  scheduling  policies); 

e.  short-range  operating  decisions,  including  de¬ 
tailed  routing  and  scheduling,  and  assignment 
of  vehicles. 

Application  to  strategic  mobility  analysis 

Long-range  options  include:  changes  in  terminal 
locations  and  capabilities;  changes  in  locations  and 
capabilities  of  origins,  including  home  stations  and 
depots;  ehanges  in  network  configurations  and  link 
capabilities;  ineluding  construction  of  new  transporta¬ 
tion  links;  ehanges  in  numbers,  types,  and  operating 
characteristics  of  aireraft,  ships,  railroad  ears  and 
other  vehicles;  research  and  development  of  new 
transportation  technologies  ineluding  vehicles, 
facilities,  and  networks. 

Mid-range  operating  policies  inelude:  changes  in 
the  numbers  of  aireraft,  ships  or  other  vehicles  al¬ 
located  to  a  theater;  ehanges  in  readiness  status  and 
mobilization  rates;  changes  in  routing  and  scheduling 
criteria  and  other  operating  policies. 

Short-run  operating  options  inelude:  routing  and 
scheduling  of  speeifie  movements,  assignments  of 
movement  priorities,  and  detailed  decisions  at  the 


installation,  terminal  and  transportation  operator 
levels. 

Strategic  mobility  models  must  have  provision  for 
analysts  to  express  particular  options  of  all  these 
types.  The  models  will  be  used  to  explore  alternative 
options,  and  so  should  be  designed  explicitly  for  ease 
of  use  for  these  purposes.  This  does  not  mean  that 
every  model  must  incorporate  the  full  set  of  options, 
but  that  every  option  must  be  provided  for  in  at  least 
one  model  in  the  set  available. 

Some  of  these  options  are  illustrated  in  Figure  8. 

Principle  VI 

In  addition  to  direct  transportation  decision  vari¬ 
ables,  transportation-related  decision  variables  must 
be  considered,  particularly  those  variables  which 
can  influence  directly  or  indireetly  the  demand  for 
transportation  (distribution  of  demand  over  space, 
over  time,  and/or  by  type  of  transportation  resources, 
and  level  of  service  required). 

Application  to  strategic  mobility  analysis 

Non-transportation  factors  which  influence  the 
demand  for  strategic  transportation  include:  uses 
of  prepositioning  and  forward  bases;  changes  in  initial 
locations  and  readiness  status  of  movement  require¬ 
ments;  ehanges  in  consumption  rates;  ehanges  in 
composition  of  the  forees  in  the  theater  or  in  tactics; 
use  of  local  products  as  opposed  to  items  moved  from 
the  CONUS;  and  ability  to  do  without  speeifie  kinds 
of  items  and/or  forees. 

This  principle  implies  that  in  addition  to  various 
transportation  models,  there  must  also  be  models  and 
procedures  for  expressing  the  indicated  options.  For 
example:  ehanges  in  movement  requirements,  in  loca¬ 
tions  and  readiness  status  of  requirements;  etc. 

Principle  VII 

The  full  set  of  consequences  of  alternative  trans¬ 
portation  systems  policies  must  be  considered,  in¬ 
cluding: 

a.  dollar  eosts  of  capital  investments  in  transporta¬ 
tion  vehicles  and  facilities; 

b.  dollar-valued  operating  eosts; 

e.  dollar-valued  ehanges  in  eosts  born  by  users  of 
the  transportation  system; 

d.  Non-dollar-valued  costs  born  by  users  of  the  sys¬ 
tem  (such  as  transit  time,  comfort,  convenience, 
flexibility,  reliability,  vulnerability,  effective¬ 
ness); 

e.  non-dollar-valued  costs  and  benefits  incurred 
by  non-users  of  the  system; 

f.  other  non-dollar-valued  or  non-quantifiable  as¬ 
pects  of  the  impaets  of  transportation  alterna¬ 
tives. 
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Application  to  strategic  mobility  analysis 

Capital  investments  in  fixed  facilities  ineluding 
networks  and  terminals  and  in  vehicles  are  direct  and 
obvious,  as  are  also  operating  cost  components  such 
as  fuel,  maintenance,  personnel,  ete.  Dollar  costs  to 
the  users  inelude  direet  transportation  costs,  as  well 
as  sueh  indirect  faetors  as  quantities  of  goods  in  the 
pipeline;  deterioration  or  wear  and  tear  of  goods; 
eost  of  distribution  systems,  including  warehouses, 
inventory  levels  and  reserve  stocks.  Non-dollar- 
valued  impacts  on  the  users  of  the  system  are  pri¬ 
marily  aspeets  of  the  military  effectiveness  of  per¬ 
sonnel  and  material,  as  affected  by  transportation 
time,  reliability,  vulnerability,  ete.  Non-dollar-valued 
impaets  upon  non-users  inelude  effects  on  the  civilian 
eeonomy  (CONUS  or  theater)  of  the  military  use 
of  transportation  resources.  The  broader  non-quanti- 
fiable  aspeets  of  impacts  are  the  over-all  vulnerability, 
reliability  and  political  desirability  of  the  transporta¬ 
tion  alternatives. 

The  strategic  mobility  criteria  in  Figure  4  sum¬ 
marize  the  major  relevant  impaets. 

The  purpose  of  a  model  is  to  predict  the  perfor¬ 
mance  of  specified  options  with  regard  to  relevant 
criteria.  Strategic  mobility  models  must  produce  in¬ 
formation  appropriate  for  predicting  the  performance 
of  options  described  above  with  respect  to  criteria 
concerning  these  impacts. 

Principal  V//I 

A  transportation  system  is  a  particular  form  of 
“market,”  in  which  supply  and  demand  reach  equilib¬ 
rium  within  the  constraining  channels  of  the  trans¬ 
portation  network. 

a.  A  number  of  “level  of  serviee”  variables  are 
neeessary  to  define  the  interrelation  of  supply 
and  demand. 

b.  The  volume  and  composition  of  the  demand 
for  transportation  depend  upon  the  level  of 
serviee  at  which  transportation  is  supplied. 

e.  The  “supply”  of  transportation,  represented  by 
the  level  of  serviee  provided,  depends  (forgiven 
resource  inputs)  upon  the  volume  and  composi¬ 
tion  of  the  demand. 

o.  Determining  the  level  of  serviee  at  which  supply 
and  demand  are  in  equilibrium  in  a  particular 
eontext  is  usually  computationally  difficult,  be¬ 
cause  of  the  complexities  of  the  transportation 
network  and  of  the  transportation  demands. 

Application  to  strategic  mobility  analysis 

The  basic  “level-of-serviee”  variables  in  strategic 
mobility  are  identified  in  Figure  1 1.  For  fixed  trans¬ 
portation  resources,  the  level  of  service,  as  measured 


in  total  trip  time,  for  example,  will  depend  upon  the 
volume  of  movements  through  the  system.  For  a  wide 
range  of  low  volumes,  travel  time  is  relatively  con¬ 
stant,  but  as  the  level  of  movement  through  the  sys¬ 
tem  increases  toward  capacity,  travel  time  increases 
rapidly  — i.e.,  the  level  of  serviee  depends  on  de¬ 
mand. 

Time 

Total  trip  time 

Reliability —  frequency  distribution  of  trip 
times 

Time  spent  at  transfer  points 
Aetual  arrival  versus  desired  arrival 
Aetual  departure  versus  ready  date 
Closure  dates  of  personnel,  equipment,  and 
total  unit 

Cost 

Operating  costs 
Fixed  costs 

Safety 

Probability  of  fatality 

Probability  distribution  of  accident  types 

Comfort  and  Convenience 
Physical  comfort 
Psychological  comfort 

Figure  1 1  —Lcvel-of-service  variables 

Usual  military  practice  is  to  establish  “firm” 
movement  requirements  — that  is,  to  fix  demand. 
The  level  of  requirements  is  generally  (but  not  al¬ 
ways)  based  upon  some  initial  broad  estimate  of  the 
demand  which  can  be  satisfied  at  an  acceptable  level 
of  serviee— for  example,  an  estimate  of  the  forees 
whieh  can  be  deployed  in  a  specified  time  period.  If 
upon  detailed  analysis  it  is  determined  that  the  level 
of  serviee  is  unacceptable  (sueh  as  movement  times 
too  great),  or  alternatively  that  there  is  exeess  capacity 
in  the  system,  then  the  level  of  requirements  will  be 
adjusted.  Although  demand  is  controlled  by  military 
planners,  it  is  a  function  of  the  level  of  serviee  sup¬ 
plied. 

Given  a  level  of  demand,  represented  by  the  set  of 
movement  requirements,  determining  the  level  of 
serviee  is  computationally  difficult.  It  is  neeessary 
either  to  do  detailed  routing  and  scheduling  of  move¬ 
ments,  or  to  use  some  aggregate  flow  model  whieh 
refleets  the  way  routing  and  scheduling  is  assumed  to 
be  done. 

Models  used  in  strategic  mobility  analysis  must 
address  explicitly  these  “market”  aspeets.  The  models 
must  be  able  to  prediet  the  various  level-of-serviee 
characteristics,  sueh  as  travel  time,  elosure  dates, 
etc.,  of  interest  to  the  planner,  for  given  options  for 
applying  the  resources  against  the  requirements. 
The  models  must  be  able  to  indicate  to  the  planner 
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potential  changes  in  his  options  which  will  result  in  a 
more  satisfactory  level  of  service.  The  models  must 
have  provisions  for  the  planner  to  revise  his  trans¬ 
portation  and  transportation-related  options  to  im¬ 
prove  the  level  of  service.  Because  supply  and  demand 
interact  within  the  transportation  network,  models 
are  necessary  to  provide  satisfactory  core  capability 
for  these  functions. 

Principle  IX 

Comparative  analyses  of  alternative  transporta¬ 
tion  systems  policies  must  maintain  the  same  set  of 
assumed  conditions.  In  particular,  the  same  volume 
of  demand  must  be  assumed  for  transportation  sys¬ 
tem  alternatives,  or  else  explicit  correction  for 
changes  in  the  level  of  demand  must  be  made. 


Application  to  strategic  mobility  analysis 
The  basic  implication  of  this  principle  is  that 
assumptions  must  be  stated  explicitly,  and  that  al¬ 
ternative  analyses  must  be  consistent.  The  key  as¬ 
sumptions  in  strategic  mobility  are  shown  in  Figure 
12. 


Readiness  status  —movement  requirements 

(units,  supplies) 

—  transportation  resources 

Network  limitations  —overflight  restrictions 

—  canal  closures 


Strategic  Warning  Time 

Enemy  actions  —hostile  actions  enroute 

-intra-theater 
—  sabotage  in  CONUS 


Weather 
Local  populace 


Supply  factors 

Movement  capability 
factors 


—  labor  availability 

—  hospitility 

—  transport  capability 

required  to  support 

—  consumption  rates 

—  actual  supply  levels 

—  maintenance  require¬ 
ments 

—  aircraft  utilization 

rates 

—  vehicle  speeds 

—  port  clearance  times 


Reliability  of  transportation  intelligence 
Figure  12  —  Strategic  mobility  assumptions 

The  implications  of  consistency  are  best  illustrated 
by  an  example.  Consider  the  problem  of  force  pro¬ 
gramming,  where  the  issue  is  which  alternative  level 
of  transportation  force  structure  to  select.  Assume 
the  alternatives  have  significant  differences  in  move¬ 
ment  capability.  To  analyze  the  alternatives,  the  level 
of  movement  requirements  must  remain  the  same  for 


each  alternative  force  structure.  Otherwise,  evaluation 
of  the  alternative  force  structures  must  balance  the 
cost  of  each  alternative  transportation  force  level 
against  the  costs  and  benefits  of  each  of  the  different 
levels  of  movement  requirements  satisfied. 

Principle  X 

Transportation  is  not  an  end  in  itself. 

Application  to  strategic  mobility  analysis 

The  ultimate  concern  of  strategic  mobility  is  the 
effectiveness  of  the  strategic  response  to  an  actual 
or  potential  aggressor.  There  are  many  alternatives 
to  transportation,  including  prepositioning,  changes 
in  force  requirements,  changes  in  movement  char¬ 
acteristics  of  equipment,  etc.  (See  Principle  VI). 
The  strategic  planner  must 

The  strategic  planner  must  continually  evaluate 
whether  transportation  is  in  fact  the  most  effective 
means  of  response. 

More  directly,  this  principle  also  means  that  the 
mobility  planner  must  continually  verify  that  in  his 
attempt  to  achieve  maximum  transportation  effective¬ 
ness,  he  does  not  reduce  over-all  strategic  effective¬ 
ness.  For  example:  maximum  utilization  of  transporta¬ 
tion  capability  may  require  excessive  travel  time  for 
personnel;  or  major  disparities  between  arrival  times 
of  personnel  and  equipment  of  units;  or  an  excessive 
grouping  of  lift  vehicles  or  saturation  of  facilities, 
thus  increasing  vulnerability  to  enemy  actions. 

B.  Alternative  levels  of  detail 

In  this  section,  we  expand  upon  Principle  1.  This 
principle  states  that  transportation  systems  consist 
of  networks  and  vehicles  flowing  through  those  net¬ 
works.  Thus,  this  principle  establishes  the  most  de¬ 
tailed  level  of  modelling  required,  in  which  every  ve¬ 
hicle  is  moved  as  an  element  over  each  link  of  the 
transportation  network  on  its  route.  This  detailed 
level  provides  a  reference  point  against  which  to  dis¬ 
cuss  other  levels. 

The  key  elements  in  modelling  a  transportation 
system  are: 

a.  the  transportation  network  — links  connected  at 
nodes  in  a  relatively  complex  fashion  to  form  a 
network; 

(1)  links  — highways,  sea  routes,  air  routes,  rail¬ 
roads,  canals,  etc. 

(2)  nodes  — may  be  enroute  terminals  — for  ex¬ 
ample,  an  enroute  air  terminal;  may  be 
storage  points  — for  example,  a  holding  and 
reconsignment  point  or  staging  area;  or  may 
be  transshipment  points  or  interchange 
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points  between  modes  — for  example,  ports 
of  embarkation  and  ports  of  debarkation; 

b.  transportation  vehicles  — differ  in  their  load 
carrying  characteristics  (for  example,  bulk 
load  capability  versus  outsize  load  capability; 
cube  capability  versus  heavy-lift  capability, 
etc.)  and  in  their  operating  characteristics  (speed, 
range,  requirement  for  specialized  loading  equip¬ 
ment,  shallow  water  draft,  etc.): 

c.  movement  requirements  — movement  require¬ 
ments  consisting  of  personnel  and  cargo  may 
vary  significantly  in  the  characteristics  which 
determine  what  vehicles  can  carry  them  and  how 
they  will  move  through  transshipment  points; 
for  example:  weight,  cube,  pallet  size,  clearance 
dimensions,  whether  heavy-lift,  and  other  char¬ 
acteristics  of  loads  are  extremely  important  in 

determining  their  impact  on  the  transportation 
system. 

The  basic  problem  to  be  addressed  in  strategic 
mobility  analysis  is  the  assignment  of  resources 
against  requirements,  as  described  by  the  three- 
cornered  triangle.  From  a  computer-systems  point 
of  view,  we  can  replace  the  requirements  and  re¬ 
sources  comers  of  the  triangle  by  two  corresponding 
files  of  data: 

a.  a  complete  set  of  movement  requirements,  de¬ 
fined  in  detail  (origin,  destination,  date  required 
at  destination,  date  ready  to  depart  origin,  move¬ 
ment  dimensions,  etc.)  as  appropriate  for  the 
specified  analysis; 

b.  a  file  of  transportation  resources  also  defined 
in  appropriate  detail  (number  of  vehicles  by 
type,  initial  position,  speed,  load  carrying  capa¬ 
bilities,  initial  availability  time;  transportation 
network  capabilities,  links,  terminals,  etc.). 

The  assignment  of  resources  against  requirements 
can  take  place  at  a  number  of  different  levels  of 
detail: 

a.  ton  miles  of  capabilities  against  ton  miles  of 
requirements  (in  general,  or  broken  down  by 
channel;  geographic  area;  passengers,  short 
tons  bulk  and  short  tons  outsize;  and/or  time 
period); 

b.  tons  per  day  per  channel,  by  gross  types  of 
movement  requirement  (pax,  short  tons  bulk, 
short  tons  outsize); 

c.  so  many  vehicles  per  day  capability  over  each 
route  — this  implies  being  able  to  do  routing  and 
scheduling  of  vehicles  in  an  approximate  man¬ 
ner; 

d.  detailed  assignment  of  requirements  against 
capabilities  — assigning  movement  units  to 


CONUS  modes  and  vehicles,  ports  of  embar¬ 
kation,  vessels  and  aircraft,  etc.,  constructing  a 
detailed,  comprehensive  movement  schedule. 

The  relations  between  levels  of  detail  are  illustrated 
in  Figure  13.  A  basic  problem  in  designing  a  system 
for  strategic  mobility  analysis  is  to  establish  the  dif¬ 
ferent  levels  of  detail  required  for  stating  the  move¬ 
ment  requirements,  for  describing  the  movement  re¬ 
sources,  and  for  analyzing  the  relationship  of  require¬ 
ments  against  resources.  Obviously,  the  level  ot 
detail  used  for  requirements  should  be  approximately 
the  same  as  that  used  for  capabilities,  in  any  particular 
analysis.  The  fundamental  question  is  what  alternative 
levels  of  detail  to  provide. 


Alternative  forms  of 
expressing  requirements: 


Gross  tonnages,  pax 
Type  forces. 


gregate  Requirement 


"Standard"  or  preselected 
mixes  of  units  (e.g.,  plans 

Detailed  selection  of  units 


Alternative  forms  of 
expressing  transportation 
resources : 


Detailed 

Requirement 


Gross  models 
(flow  model, 
linear  program¬ 
ming,  gross 
feasibility 
estimator ) 


General  resources  . 


/ 


Simulation 


Aggregate  (channel) 
capabilities  and 
productivities 


Specific  resources 
(vehicles  by  type, 
detail  of  network) 


Detailed 

model  (especially 
simulation) 


Figure  1 3  —Conceptual  relationships  between  aggregate  and 
detailed  levels 

Corresponding  to  the  different  levels  of  detail  in 
which  requirements  and  resources  Files  can  be  ex¬ 
pressed,  are  different  levels  of  models  of  the  trans¬ 
portation  system: 

a.  Total  throughput  capability:  in  this,  the  most  ag¬ 
gregate  level,  transportation  resources  are  repre¬ 
sented  by  their  total  throughput  capability,  such  as 
three  thousand  ton  miles  per  day.  The  image  cor¬ 
responding  to  this  model  is  of  a  pipe  through  which 
movement  requirements  can  be  passed  at  the  in¬ 
dicated  rate.  The  extent  to  which  the  indicated 
rate  takes  into  account  all  the  subtlety  of  trans¬ 
portation  system  elements  determines  the  relia¬ 
bility  of  this  kind  of  model.  In  particular,  for  highly 
complex  networks  in  which  a  variety  of  flows  occur 
more  or  less  simultaneously,  and  in  which  the  flows 
have  diverse  patterns  of  origins  and  destinations 
and  have  diverse  movement  characteristics,  the 
reliability  of  this  “throughput”  model  is  highly 
questionable.  On  the  other  hand,  a  total  throughput 
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model  may  well  be  appropriate  for  such  steady- 
state  flow  problems  as  represented  by  flow  over  a 
very  small  number  of  parallel,  non-interacting  routes 
for  airlift  to  a  fairly  isolated  theater,  in  which  the 
number  of  vehicles  available  is  relatively  well  with¬ 
in  the  capability  of  enroute  support  facilities  and 
in  which  the  vehicles  aro  rpCyCied  at  a  steady  rate. 

b.  Network  flow  model,  in  this  type  of  model,  the 
transportation  system  is  represented  by  a  net¬ 
work  in  which  eaeh  link  has  a  capacity,  such  as 
short  tons  per  day.  One  example  is  the  Ford-Fulker- 
son  flow  model  (cf.  Ford  and  Fulkerson,  1962). 
This  takes  into  account  the  interacting  or  topo¬ 
logical  eharaeteristies  of  transportation  networks 
and  so  is  somewhat  more  realistic  than  the  gross 
throughput  model.  However,  like  the  throughput 
model,  it  suffers  from  assumptions  about  vehicle 
capabilities  necessary  in  deriving  the  channel 
capacities. 

c.  Vehicle  recycling  model:  in  vehicle  recycling 
models,  transportation  networks  are  ignored  or 
represented  only  as  route  distances  and  perhaps 
route  capacities;  that  is,  the  interactions  of  a  variety 
of  routes  using  several  common  links  are  ignored. 
The  major  focus  of  the  model  is  on  taking  a  fixed 
number  of  vehicles  and  considering  the  impact  of 
their  recycle  times:  that  is,  the  time  it  takes  to  move 
them  out  to  the  origin  to  pick  up  the  load,  from  the 
origin  to  the  destination,  and  back  from  the  destina¬ 
tion  to  a  repositioning  point  for  pickup  of  the  next 
load.  An  example  of  this  is  the  Airlift  Capabilities 
and  Requirements  Estimator  (ACRE)  model  of 
the  U.S.  Military  Airlift  Command,  or  the  Airlift 
Deployment  Simulator  (AIDS)  model  of  Stanford 
Research  Institute.  These  models  take  into  ac¬ 
count  the  effects  of  vehicle  availability,  at  the  ex¬ 
pense  of  ignoring  network  interactions. 

d.  Comprehensive  transportation  model:  a  compre¬ 
hensive  transportation  model  would  take  into  ac¬ 
count  both  network  characteristics,  and  vehicle 
recycling  and  availability  characteristics.  Sueh  a 
model  might  be  obtained  initially  by  relating  the 
Ford-Fulkcrson  flow  algorithm  to  explicit  use  of 
recycling  models  such  as  ACRE.  However,  a  com¬ 
prehensive  model  should  be  developed  which  is 
explicitly  designed  to  consider  the  flow  of  discrete 
vehicles  through  transportation  networks,  con¬ 
sidering  both  the  topology  or  interaction  char¬ 
acteristics  of  the  network,  and  the  vehicle  avail¬ 
ability  and  reeyeling  aspects.  Initially,  such  a  model 
might  make  fairly  gross  approximations  to  a  third 
aspect  of  the  problem,  the  loading  problem  (how 
the  detailed  components  of  movement  requirements 


are  distributed  over  or  loaded  into  the  available 

vehicles). 

We  do  not  anticipate  that  such  a  “comprehensive" 
model  can  be  developed  as  a  single  package.  Rather, 
we  expect  that  various  forms  of  “comprehensive" 
models  will  be  obtained  by  linking  together  com¬ 
ponents  of  other  types  of  models.  This  is  illustrated 
in  our  discussion  of  the  relationship  between  optimiz¬ 
ing  and  simulation  models  following. 

Channel  flow  models  and  detailed  models  lie  at 
two  extremes  of  the  spectrum  of  detail.  For  some 
kinds  of  studies,  channel  throughput  capabilities  may 
be  a  reasonable  approximation  to  detailed  networks. 
These  channels  may  be  obtained  by  aggregating  links 
and  nodes  in  a  detailed  network  and  estimating  vehicle 
utilization  rates  and  carrying  capabilities  to  achieve 
a  single  numerical  flow  capacity  for  each  channel. 
However,  to  verify  such  an  approximation,  and  for 
detailed  analysis  which  explicitly  considers  the  in¬ 
teraction  of  different  modes  of  transportation  and  the 
effects  of  terminal  or  interchange  capabilities,  trans¬ 
portation  must  be  modelled  in  a  way  which  takes  into 
account  the  movements  of  discrete  vehicles  singly 
or  in  units  through  the  detailed  topological  structure 
of  the  flow  network. 

Thus,  the  levels  of  detail  in  a  strategic  mobility 
analysis  system  will  range  from  detailed  movement 
scheduling  models  which  consider  discrete  vehicles, 
to  aggregate  or  gross  flow  models  which  consider 
only  channel  flow  capacities. 

C.  Need  for  a  variety  of  models 

The  overall  implications  of  the  preceding  discus¬ 
sion  are  that  there  is  no  single  model  adequate  for 
all  strategic  mobility  analyses.  Because  of  the  wide 
variety  of  problem  types  and  modelling  issues,  there 
must  be  a  variety  of  models  and  techniques  available. 
These  models  will  differ  in  other  aspeets  in  addition 
to  level  of  detail. 

Other  basic  differences  among  models  are. 

a.  the  eost  of  using  a  model  — in  time  and/or  dollars: 

( 1 )  initial  set-up  cost; 

(2)  cost  for  each  successive  application; 

b.  accuracy  or  validity-probability  that  the  model 
gives  the  right  decision;  reasonableness  of  model 
as  representation  of  the  real  world; 

c.  scope  — what  planning  options  and  performance 
measures  are  within  the  scope  of  the  model; 

d.  sensitivity —  degree  to  which  results  from  the 
model  vary  with  uncertainty  in  key  data; 


*For  a  theorclical  lrealment  of  ihc  roles  of  models  in  a  problem¬ 
solving  process,  see  Manheim  (1966a). 
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e.  degree  of  “optimality”  — extent  to  whieh  model 
produces  a  solution  which  is  optimal  with  respeet 
to  some  criterion. 

A  strategic  mobility  analysis  system  must  have  a 
variety  of  models,  differing  in  detail,  in  degree  of 
optimality  of  the  solution,  in  coverage  of  the  prob¬ 
lem,  in  level  of  computational  eost,  and  in  other  re- 
speets.  Every  model  has  its  advantages  and  disad¬ 
vantages.  These  models  should  be  available  in  a  single 
system  environment,  with  routines  to  allow  as  com¬ 
plete  compatibility  as  possible  sueh  that  the  models 
all  operate  off  the  same  data  base.  Only  after  extensive 
operational  experience  is  acquired  with  a  full  set  of 
models,  will  it  be  possible  to  really  identify  the  rela¬ 
tive  roles  of  different  models.  Until  then,  the  analyst 
must  have  freedom  to  experiment  with  using  alterna¬ 
tive  models  in  a  flexible  environment.* 

There  should  be  one  basic  set  of  data  files  in  a 
strategic  mobility  analysis  system.  That  is,  any  routine 
whieh  requires  files  in  a  non-standard  format  will 
have  to  be  preceded  by  pre-proeessing  routines  which 
eonvert  data  from  the  format  of  the  basie  files  to 
temporary  files  in  the  formats  equired  by  these 
routines.  This  approach,  while  perhaps  expensive  in 
computational  time,  allows  implementation  of  a  flexi¬ 
ble  environment  in  whieh  simulation  models,  linear 
programming,  and  other  algorithms  are  used  together. 

The  relationship  of  linear  programming  and  similar 
optimizing  algorithms  to  simulation  models  may  take 
several  forms,  and  illustrates  the  desired  degree  of 
flexibility  and  interaction  among  models. 

By  simulation  models  we  mean  models  whieh  may 
have  partially  optimizing  components,  for  example, 
the  assignment  of  loads  to  ships.  However,  a  simu¬ 
lation  model  is  characterized  by  the  faet  that  the 
result  is  dependent  upon  speeifie  values  of  parameters 
given  by  the  planner;  the  result  of  a  simulation  is  more 
a  prediction  of  the  eonsequenees  of  the  planner’s 
parameter  values  than  an  optimal  solution  of  a  prob¬ 
lem  given  those  parameter  values. 

In  contrast  to  simulation  models,  optimizing  models 
such  as  linear  programming  or  Ford-Fulkerson  flow 
algorithms  give  a  solution  whieh  is  optimal  within  the 
eontext  of  the  model  (whieh  may  be  extremely 
limited).  For  example,  optimizing  the  distribution  of 
flow  through  a  transportation  network  to  achieve 
maximum  total  throughput  capability;  minimizing 
the  total  cost  of  strategic  mobility  resources  to  meet 
given  requirements.  A  simulation  approach  to  this 
last  example  would  eonsist  of  the  following:  given  a 
speeifie  fleet  mix,  what  is  its  cost  and  does  it  meet 
the  movement  requirements.  Thus,  the  simulation 
model  would  have  to  be  used  many  many  times,  vary¬ 


ing  the  fleet  mix  eaeh  time,  to  determine  the  least- 
eost  fleet  mix  whieh  meets  the  requirements. 

There  is  a  role  in  a  strategic  mobility  analysis  sys¬ 
tem  for  eaeh  type  of  model.  Some  of  these  roles  are 
as  follows: 

a.  optimizing  algorithms  used  to  identify  an  optimal 
solution  at  a  general  level; 

b.  simulation  models  used  for  detailed  evaluation 
of  a  particular  solution; 

c.  simulation  models  used  to  determine  coef¬ 
ficients  for  input  to  optimizing  models  (See 
Figure  14); 

d.  optimizing  models  used  as  components  within 
simulation  models; 

e.  optimizing  models  used  to  determine  a  starting 
point  for  a  simulation  model. 

This  variety  of  potential  interactions  between  simu¬ 
lation  and  optimizing  models  justifies  the  basie  design 
decision  in  the  first  paragraph.  With  one  basie  set  of 
data  files  and  the  development  of  appropriate  data 
conversion  routines,  optimizing  and  simulation  models 
ean  be  tied  together  in  a  variety  of  ways  to  obtain 
quite  different  kinds  of  strategic  mobility  models.  It 
is  highly  important  that  the  system  have  this  capa¬ 
bility. 
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each  link  in  net¬ 
work,  as  function 
of  number  of  ve¬ 
hicles  and  mix  of 
types  assigned  to 
that  link 

'l' 

Productivities  & 
capacities 

4 

Capacity  con¬ 
straint  equations 


Maximal  flow  through 
„  total  network  (in  the 
sense  of  minimum  cut); 
identify  all  constrain 
ing  arcs;  capacity 
ranging  to  analyze 
bottleneck  patterns 


Figure  14—  Relation  between  linear  programming  and 
simulalion  models 
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D.  Implications  for  system  design 

The  basic  point  of  this  discussion  is  that  an  analysis 
capability  for  strategic  mobility  must  utilize  a  variety 
of  models.  A  corollary  argument  is  that  the  planner 
must  be  able  to  use  these  models  in  an  interrelated 
fashion.  To  amplify: 

1.  models  must  be  provided  for  analyzing  all 
modes  of  transportation, 

2.  models  must  be  provided  for  simulating  all  types 
of  movement  requirements, 

3.  models  must  be  able  to  consider  interrelations 
among  successive  modes  of  transportation  as 
requirements  flow  from  initial  origin  to  final 
destination,  and  must  specifically  address  the 
relationships  between  Hows  into  and  out  of 
terminals  and  other  mode  interchange  points, 

4.  models  must  be  able  to  analyze  the  full  set  of 
transportation  and  transportation-related  plan¬ 
ning  options,  in  all  time  frames  from  long-range 
to  current  operations, 

5.  models  must  provide  information  with  which 
the  planner  can  identify  all  the  significant  impacts 
of  the  alternatives  open  to  him, 

6.  models  must  explicitly  recognize  the  interaction 
of  supply  and  demand  in  the  transportation  mar¬ 
ket,  and  must  provide  the  planner  flexible  tools 
for  exploring  the  levels  of  service  which  can  be 
achieved  with  alternative  levels  and  mixes  of 
requirements, 

7.  models  must  allow  explicit  variation  of  key 

assumptions, 

8.  models  must  be  available  at  different  levels  of 
detail  and  with  different  costs,  emphases,  and 
coverages  of  the  options,  so  that  the  planner  can 
use  the  model  most  appropriate  to  the  analysis 
task, 

9.  models  must  be  available  in  a  common  environ¬ 
ment,  related  to  the  same  basic  data  base,  so 
that  the  planner  can  use  different  models  in  an 
interrelated  manner. 


III.  The  Systems  Analysis  Approach 

A.  Ideal  approach 

The  basic  theory  of  systems  analysis  is  shown 
diagrammatically  in  Figure  15.  Systems  analysis 
works  toward  this  theoretical  approach  as  an  objec¬ 
tive.  Under  ideal  conditions,  this  theory  implies  the 
sequence  of  analy  sis  shown  in  Figure  16. 

In  practice,  even  the  best  systems  analyses  don’t 
achieve  this  ideal.  Nevertheless,  it  provides  an  impor¬ 
tant  and  useful  conceptual  framework  (cf  Hitch  and 
McKean,  1960,  particularly  appendix  by  Enthoven; 
also  Henderson  and  Quandt,  1958),  and  an  objective 
to  guide  analyses. 


The  difficulties  in  implementing  this  ideal  approach 
in  practice  arise  for  the  following  reasons: 


a)  Production  Possibilities 


b)  Cost  Alternatives 


c)  Optimum  Production  Points 


d)  Cost-Ef feetivoness 
Function 


Cost 


F  igure  1 5  —  Systems  analysis:  theory 

a.  Description  of  space  of  alternatives; 

The  set  of  all  possible  alternatives  is  not  easily 
described  in  terms  of  a  small  number  of  continuous 
variables;  for  example,  see  the  variety  of  trans¬ 
portation  resources  identified  in  Figure  2.  Further¬ 
more,  the  spaec  of  alternatives  is  large,  and  generation 
of  a  single  reasonable  alternative  is  difficult —  for 
example,  the  generation  of  a  single  transportation 
plan  whieh  is  likely  to  be  feasible. 

b.  Measures  of  effectiveness: 

In  general,  there  is  not  a  single  major  criterion  of 
effectiveness  of  a  transportation  plan,  as  indicated  in 
Figure  4.  All  of  the  measures  shown  there,  and  others, 
are  potentially  significant  at  different  stages  of  analy  sis 
of  a  strategic  mobility  problem.  It  is  not  acceptable 
to  assume  that  all  of  these  measures  can  be  uniformly 
collapsed  into  two  measures,  one  of  effectiveness 
and  one  of  cost.  Rather,  several  different  measures 
must  be  identified  separately  and  so  carried  through 
the  analysis.  Only  under  rather  constrained  conditions 
or  with  a  narrow  scope  of  alternatives  can  a  single 
measure  be  used:  for  example,  closure  time  can  be 
used  only  when  buildup  rate,  transit  time,  unit  in¬ 
tegrity,  and  other  criteria  are  all  satisfactorily  met. 
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for  a  balanced  deployment  force  with  resupply 
assumed  in  balance. 

Define  space  of  alternatives  (x,y). 

Identify  measure  of  effectiveness  q,  and  measure 
of  cost  c. 

i 

Define  models  or  procedures  to  be  used  to  deter¬ 
mine,  for  any  alternat've  (x,y),  (1)  its  cost,  c(x,y), 
and  (2)  its  effectiveness,  q(x,y). 

1 

For  all  alternatives  (x,y),  determine  effectiveness 
q  and  cost  c. 

i 

Construct  production  possibilities  curves: 
identify  all  alternatives  which  produce  same  ef¬ 
fectiveness,  for  different  levels  of  effectiveness. 

Construct  cost  alternatives  curves:  identify  all 
alternatives  which  have  same  cost,  for  different 
levels  of  cost. 

i 

Determine  optimum  production  points:  for  each 
level  of  effectiveness,  find  that  alternative  which 
has  least  cost. 

A 

Construct  the  cost-effectiveness  function,  based 
upon  the  least-cost  alternative  for  each  level 
of  effectiveness. 

A 

By  examining  the  cost-effectiveness  function, 
select  the  optimum  alternative  as  that  one  with 
greatest  cost  for  which  the  marginal  increase  in 
effectiveness  per  unit  cost  is  greater  than  the 
minimum  acceptable  rate  of  return. 

Hgure  1 6  — Systems  analysis:  ideal  approach 

c.  Models  for  determining  cost  and  effectiveness: 

As  we  pointed  out  in  Section  III,  there  are  a  variety 

of  models  required  in  mobility  analysis.  Given  a 
particular  alternative  mobility  plan,  determination 
of  its  costs  and  effectiveness  is  difficult,  complex, 
and  expensive.  Cost  and  effectiveness  are  not  simple 
analytical  functions  of  the  alternatives. 

d.  Identification  of  all  alternatives  which  have 
same  cost  or  same  effectiveness: 

Because  of  the  variety  and  complexity  of  the  alter¬ 
natives  and  of  the  models  for  evaluating  the  alter¬ 
natives,  exploration  of  the  space  of  alternatives  to 
identify  equivalances  of  cost  and/or  effectiveness 
is  very  difficult.  A  realistic  approach  must  be  a  “trial- 


and-error”  approach:  generate  one  or  several  alter¬ 
natives,  determine  the  corresponding  costs  and 
effectiveness,  and  attempt  to  infer  from  the  limited 
set  of  alternatives  the  minimum  cost  alternative  for 
each  level  of  effectiveness.  If  desired,  return  to  gener¬ 
ate  more  alternatives. 

This  ad  hoc  approach  is  necessary  while  dealing 
with  major  alternatives.  However,  there  is  a  useful 
role  for  highly-specialized  explorations  of  equiv¬ 
alences  over  specific  aspects  of  the  alternatives: 
for  example,  small  changes  in  the  number  or  mix  of 
aircraft  applied  to  move  the  requirements;  effect  of 
alternative  locations  within  the  US  on  airlift  require¬ 
ments,  in  terms  of  additional  aircraft  required  to  meet 
same  delivery  times,  or  exploration  of  relation 
between  US  location  and  ready  date,  with  airlift 
available  held  constant.  There  are  many  such  partial 
explorations  possible,  and  the  analyst  can  get  impor¬ 
tant  insights  into  the  overall  problem  through  conduct¬ 
ing  such  tradeoff  analyses.  However,  to  do  these, 
the  majority  of  variables  must  be  held  constant  while 
those  under  focus  are  varied;  and  so  the  basic  prob¬ 
lem  of  finding  major  alternatives  with  equivalent 
costs  and/or  effectiveness  must  be  handled  in  the  ad 
hoc  approach  described  above. 

e.  Determination  of  cost-effectiveness  function: 

The  cost-effectiveness  function  represents  the  set 
of  all  alternatives  (x,y)  such  that  for  any  alternative 
in  this  set,  there  is  no  other  alternative  with  lower 
cost  for  the  same  or  greater  level  of  effectiveness,  or 
with  greater  effectiveness  for  the  same  cost.  This 
assumes  that  the  costs  and  effectiveness  of  all  alterna¬ 
tives  are  known  with  certainty,  and  so  for  all  the  rea¬ 
sons  identified  above  is  unrealistic.  The  only  feasible 
approach  in  problems  as  complex  as  strategic  mobility 
is  to  array  the  alternatives  in  order  of  increasing  cost, 
and  apply  a  dominance  check.  The  dominance  check  is 
achieved  by  considering  each  alternative  in  turn, 
checking  that  there  is  no  other  alternative  with  lower 
cost  for  the  same  effectiveness  or  with  greater 
effectiveness  for  the  same  cost.  Wherever  the  domi¬ 
nance  check  is  not  satisfied,  the  less  desirable  of  the 
two  alternatives  is  discarded. 

B)  Practical  approaches  to  systems  analysis 

These  difficulties  do  not  invalidate  the  ideal  of  sys¬ 
tems  analysis,  provided  the  analysis  approach  is 
modified  appropriately.  The  first  step  is  to  adopt  the 
sequential  process  shown  in  Figure  17.  This  process 
emphasizes  the  need  to  generate  alternatives  one  by 
one.  Further,  q  and  c  may  be  vector-valued  as 
appropriate,  to  allow  for  multiple  measures  of  effec¬ 
tiveness.  The  idea  of  a  dominance  check  is  easily 
extended  to  multiple  dimensions. 
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Define  space  of  alternatives  (x,y). 

i 

Identify  measure  of  effectiveness  q,  and  measure 
of  cost  c. 

i 

Define  models  or  procedures  to  be  used  to  de¬ 
termine,  for  any  alternative  (x,y),  (1)  its  cost, 
c(x,y),  and  (2)  its  effectiveness,  q(x,y). 

— I  .  1 

Generate  an  alternative  (x,y). 

i 

Determine  the  effectiveness  q  and  cost  c  for  the 
alternative. 

i 

Dominance  check:  Compare  alternative  with 
all  others  previously  generated.  One  alternative 
dominates  another  if  the  first  has  higher  ef¬ 
fectiveness  for  equal  or  lower  cost  than  the 
second,  or  if  the  first  has  the  same  effective¬ 
ness  but  lower  cost.  Discard  the  dominated 
alternative. 

i 

Cost-effectiveness  function:  Array  the  un¬ 
dominated  alternatives  in  order  of  increasing 
cost.  (Check  that  effectiveness  increases  with 
cost.) 

i 

By  examining  the  cost-effectiveness  function, 
select  the  optimum  alternative  as  that  one  with 
greatest  cost  for  which  the  marginal  increase 
in  effectiveness  per  unit  cost  is  greater  than  the 
minimum  acceptable  rate  of  return. 

1 

Compare  the  optimum  alternative  found  so  far 
with  the  prospective  payoffs  of  additional  analy¬ 
sis.  Decide  whether  to  terminate  analysis  or  to 
generate  another  alternative. 


Figure  1 7  —  Systems  analysis:  sequential  approach 

The  second  possibility  is  to  extend  the  idea  of 
sequential  analysis  to  multiple  levels  of  detail.* 
A  highly  simplified  example  will  illustrate  this 
approach. 

Consider  a  deployment  in  which  only  one  type  of 
aircraft  and  one  type  of  ship  are  available.  The  alter¬ 
natives  can  be  expressed  as:  number  of  aircraft, 
allowable  load  per  aircraft  in  tons  per  day,  number 
of  ships,  and  capacity  per  ship  per  day.  (Assume  that 
the  load  capacities  have  already  been  scaled  appro¬ 
priately  to  tons  per  day  throughput  over  the  specified 
routes.)  The  alternatives  can  also  be  represented  in 

*For  a  Bayesian  Decision  Theory  model  of  this  process,  sec  Man 
heim  (1966a). 


another  way:  throughput  by  air  and  throughput  by 
sea  (per  day).  Air  throughput  is  calculated  as  the 
number  of  aircraft  times  the  allowable  load  per  day 
per  aircraft.  Sea  throughput  is  the  number  of  ships 
times  capacity  per  day. 

For  a  given  force  to  be  deployed,  the  problem  is 
to  find  the  cost-effectiveness  function  for  the  alter¬ 
natives.  The  alternatives  can  be  treated  at  two  levels 
of  detail.  The  “gross”  treats  alternatives  defined  in 
the  two  variables,  air  throughput  and  sea  throughput. 
The  “detailed”  level  treats  alternatives  defined  in  the 
four  variables,  number  of  aircraft,  aircraft  load,  nu  - 
ber  of  ships,  ship  load.  With  appropriate  models,  cost 
and  effectiveness  can  be  determined  at  either  the 
gross  or  detailed  levels. 

The  important  point  is  that  each  gross  alternative 
corresponds  to  a  large  number  of  detailed  alter¬ 
natives:  for  example,  an  airlift  throughput  capa¬ 
bility  of  1000  tons  per  day  can  be  achieved  by  50 
planes  each  carrying  20  tons  per  day,  25  planes 
carrying  40  tons  per  day,  or  40  planes  carrying  25 
tons  per  day.  Therefore,  the  costs  and  effectiveness 
of  gross  alternatives  can  only  be  estimated  roughly; 
and  only  for  detailed  alternatives  can  accurrate  costs 
and  effectiveness  be  determined.  However,  the 
models  for  evaluating  gross  alternatives  may  be  much 
easier  and  cheaper  (time,  cost,  data,  etc.)  to  use  than 
the  detailed  models.  Therefore,  the  planner  will  want 
to  use  both  levels  of  detail  in  a  balanced  way;  he  will 
use  the  gross  level  of  analysis  to  identify  the  general 
shape  of  the  cost-effectiveness  function,  and  will  use 
the  detailed  level  to  determine  precise  costs  and 
effectiveness  for  detailed  alternatives  of  most  interest. 

The  general  methodology  for  doing  systems  analysis 
in  this  “hierarchically  structured”  sequential  manner 
is  shown  in  Figure  1 8. 

C.  Implication  for  system  design 

The  systems  analysis  ideal  provides  a  general 
framework  of  analysis  which  is  not  realizable  in 
practice.  Design  of  an  analysis  capability  must 
recognize  that  this  framework  is  an  objective,  but 
that  practical  analysis  of  real  problems  will  fall  short 
of  the  objective.  This  implies  that  the  analysis  capa¬ 
bility  must: 

a.  allow  explicit  generation  of  large  numbers  of 
alternatives; 

b.  provide  rapid  evaluation  of  alternatives  to 
eliminate  quickly  those  which  are  likely  to  be 
dominated; 

c.  provide  detailed  evaluation  of  those  alter¬ 
natives  remaining; 

d.  allow  planner  control  of  explorations  of  trade¬ 
offs  among  small  sets  of  variables; 
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Define  space  of  alternatives  (x,y). 

1 

Identify  measure  of  effectiveness  q,  and  measure  of 
cost  c. 

4 

Detine  two  sets  of  models  for  determining  cost  and 
effectiveness  of  alternatives:  (I)  one  set  for  precise 
evaluation  of  detailed  alternatives  (x,y),  (2)  another 
set  for  approximate  evaluation  of  gross  alternatives, 
or  subsets  {(x,y)} 

-  4 

Determine  whether  to  operate  at  gross  or  detailed 
levels, 

4 

i - 

(Detailed  level) 

i 

Generate  a  detailed 
alternative  (x,y). 

i 

Determine  the  effectiveness 
q  and  cost  c  for  the  alterna¬ 
tive,  using  the  detailed 
models. 

i 

If  this  alternative  can  be 
considered  to  be  a  detailed 
version  of  some  gross 
alternative,  use  (q,c) 
evaluation  of  detailed  al¬ 
ternative  to  revise  esti¬ 
mates  of  ({q,c})  for  that 
gross  alternative. 

1 

Dominance  check:  Com¬ 
pare  this  alternative  with 
other  detailed  alternatives. 

Discard  dominated  al- 


ternatives. 

i 

Cos,-  / 

By  examining  the 

cost-effective- 

effectiveness 

ness  function,  select  the  optimum 

function: 

detailed  alternative. 

array  the  un- 

i 

dominated 

Compare  the  optimum  detailed 

detailed 

alternative  found  so  far  with  the 

alternatives 

prospective  payoffs  of  additional 

in  order  of 

analysis.  Decide  whether  to  termi¬ 

increasing 

nate  analysis  or  generate  another 

cost. 

alternative.  4 

higure  1 8 -Systems  analysis:  hierarchical  structure  (two-level) 


e.  provide  “record-keeping”  facilities  to  allow  the 
analyst  to  store  the  results  of  his  analysis  in  a 
way  amenable  to  the  systems  analysis  frame¬ 
work. 

f.  allow  the  planner  flexibility  in  using  models 
of  different  levels  of  detail  at  different  points 
in  the  analysis  process  (cf.  Manheim,  1967). 

IV.  Analysis  Environment 

A,  Relations  between  planner  and  system 
In  the  preceding  sections,  wc  have  indicated  the 
arguments  for  a  highly  flexible  interactive  relation¬ 
ship  between  the  planner  and  the  ADP  system.  The 
purpose  of  this  section  is  to  briefly  describe  how  this 
can  be  achieved  with  current  computer  technology. 

The  analysis  tasks  will  range  over  a  wide  spectrum 
in  terms  of  their  requirements  on  the  computer  system 
(Figure  19.)  There  will  be  many  times  when  a  planner 
simply  wishes  to  initiate  the  running  of  a  large  com¬ 
putational  routine,  such  as  a  linear  programming 
or  network  flow  model,  or  a  complex  simulation 
model.  There  will  be  other  times  when  the  planner 
will  want  to  interact  very  rapidly  and  adaptively: 
selecting  items  out  of  one  file  and  ordering  them  in 
another,  requesting  subtotals  of  various  kinds,  and 
performing  a  variety  of  manipulations  of  the  data. 
In  the  first  sense,  the  planner  will  be  using  the  com¬ 
puter  much  as  he  has  traditionally  used  it:  he  would 
approach  the  computer  only  to  initiate  a  large  pro¬ 
cessing  task,  and  then  again  only  to  receive  the  results. 
In  the  latter  case,  the  planner  would  be  using  the 
computer  as  almost  a  sketch  pad  or  notebook,  inter¬ 
acting  with  it  as  he  would  with  a  pencil  and  ordinary 
paper  tablet. 

The  majority  of  planner  analysis  tasks  will  be 
between  these  two  extremes  of  the  spectrum.  A 
typical  analysis  sequence  will  probably  be  a  mixture 
of  quick,  highly  interactive  data  manipulations: 
selection  of  alternative  processing  routines;  ex¬ 
ecution  of  processing  routines  of  short,  medium,  and 
occasionally  significant  length;  the  monitoring  of 
those  routines,  with  consequent  changes  in  pa¬ 
rameters  as  they  are  executed;  and  display  of  results. 

T  he  basic  operations  of  the  analyst  in  this  system 
will  be  using  programs  and  procedures  available  to 
formulate  alternative  movement  plans,  and  to  analyze 
those  plans.  As  pointed  out  in  Section  II,  all  of  the 
analysis  tasks,  whether  concerned  with  vehicle  R&D 
or  network  construction,  would  involve  the  same 
basic  function,  balancing  requirements  against  re¬ 
sources.  But  in  addition  to  executing  this  function 
many  many  times,  the  analyst  will  also  wish  to  explore 
alternative  movement  planning  procedures,  such  as 
constructing  planning  procedures  and  larger  models 
out  of  component  models  and  procedures.  The  planner 


(Gross  level) 

i 

Generate  a  gross 
alternative  {(x,y)}, 

1 

Estimate  the  approxi- 
nate  range  of  effective¬ 
ness  and  cost  ({q,c}), 
for  this  alternative  using 
the  gross  model. 

4 

Dominance  cheek: 
compare  this  alternative 
with  other  gross  alter¬ 
natives,  Discard  domi¬ 
nated  alternative  (tem¬ 
porarily  because  later 
detailed  alternatives 
may  cause  revision  of 
estimate  {(q,c)}). 
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will  also  be  concerned  with  routine  and  semi-routine 
maintenance  of  the  data  base,  including  review  and 
familiarization. 

Data  Access  and  Manipulations 

Review  of  file  contents  (display,  print)  in  variety 
of  formats 

Addition/deletion/revision  of  data 
Creation  of  new  file(s)  from  one  or  more  files  — 
merge,  extraction 

Change  of  file  formats  and  sequences 

Execution  of  Computations 
Set-up  — designation  of  data  files,  parameter 
values,  computational  options 
Actual  calculations  — short  bursts,  long  sequences 
Monitoring  long  runs  and  modification  of  cal¬ 
culations. 

Review  of  results 

Construction  of  New  Computational  Procedures 
I  inking  of  modules  and  naming  of  composite 
sequence 

Designation  of  conventions  for  data  files,  pa¬ 
rameter  values,  computational  options 

Construction  of  Modules 
Program  ming 
Debugging 
Compilation 
Testing 

Acceptance  into  repertory 

Systems  Programming 

Hgure  19  —  Modes  of  use  of  the  system 

There  w  ill  also  be  a  need  for  use  of  the  system  as  an 
education  and  training  device:  some  routines  in  the 
system  will  be  designed  explicitly  for  programmed 
instruction  in  the  use  of  the  system. 

In  design  and  use  of  a  system  such  as  this,  a  carc- 
ful  balance  between  roles  of  man  and  machine  must 
be  maintained.  Wherever  possible,  the  planner 
should  be  forced  to  make  explicit  judgments  about 
parameter  values,  criteria,  etc,,  as  opposed  to  building 
into  the  logic  of  a  program  a  particular  set  of  criteria 
and  a  particular  scheduling  approach.  (However, 
“standard”  values  would  be  provided  so  that  planner 
need  operate  only  on  “exception”  basis  to  override 
“standard”  values.)  This  serves  several  purposes: 
first,  it  forces  the  planner  to  think  about  the  problem 
and  to  become  involved  with  its  details,  thus  trying 
to  impress  upon  him  that  he  should  not  treat  the  com¬ 
puter  answer  as  dogma.  Second,  it  assists  in  pre¬ 
venting,  to  the  maximum  extent  possible,  the  insti¬ 
tutionalization  of  the  particular  prejudices  of  one  or 
several  planners  or  programmers, 

B)  Basie  technological  capabilities 

The  basic  computer  capability  necessary  to  provide 


the  highly  interactive  support  required  is  the  kind  of 
system  now  known  as  a  multi-user,  remote-access, 
“time-sharing”  system.  Through  sharing  the  funda¬ 
mental  processing  capabilities  of  the  computer  simul¬ 
taneously  among  a  large  number  of  users,  this  kind 
of  capability  can  provide  access,  through  consoles 
at  remote  locations,  to  a  large  number  of  individuals. 
Each  individual  can  utilize  the  computer  in  a  highly 
interactive  way,  having  the  eomputer  respond  to  him 
about  as  rapidly  as  he  can  absorb  the  information. 
For  processing  tasks  which  require  significant 
amounts  of  computation,  the  user  does  not  need  to 
stay  at  his  input-output  device,  but  need  only  initiate 
the  processing  and  return  to  receive  the  results;  how¬ 
ever,  if  he  wishes,  he  may  monitor  the  processing, 
review  intermediate  results,  and  make  parameter 
changes. 

These  systems  will  have  a  variety  of  interactive 
capabilities,  including  flexible  problem-oriented 
languages  which  allow  the  planner  to  utilize  the  com¬ 
puter  system  very  rapidly.  Such  a  computer  system 
can  approach  the  ideal  of  an  electronic  “writing  tablet” 
or  “sketch  pad,”  in  which  the  computer  provides 
services  to  the  user  as  rapidly  as  he  can  formulate 
his  requirements. 

Remote  access  consoles  will  be  a  teletypewriter  or 
other  similar  capability,  and  might  also  have  graphic 
displays  and  local  capabilities  for  transmitting  and 
receiving  facsimile  reproductions.  In  addition,  high¬ 
speed  bulk  printing  capability  will  be  provided  for 
producing  large,  voluminous  reports  and  details,  and 
permanent  records  of  analyses. 

Such  technological  capabilities  are  available  and 
are  technologically  feasible  for  implementation  at 
this  time.* 

C.  Conclusions 

The  previous  sections  of  this  paper  have  established 
the  requirement  for  highly  flexible  interactive  pro¬ 
cessing,  in  order  to  effectively  accomplish  strategic 
mobility  systems  analysis.  Such  capability  is  available 
through  multi-user,  on-line,  remote-access  systems. 
These  systems  will  provide  a  highly  responsive 
capability,  allowing  the  planner  to  explore  tradeoffs; 
to  explore  alternative  plans,  including  alternative 
levels  of  movement  requirements  and  resources; 
to  explore  the  impact  of  assumptions;  and  in  general 
to  become  highly  familiar  with  the  shape  of  his  prob¬ 
lem,  the  alternatives  open  to  him,  and  the  issues. 

With  this  kind  of  system,  there  is  little  danger  that 
the  machine  will  replace  the  planner:  this  kind  of 

*See  for  example  articles  on  Project  MAC  and  Sysicm  Develop- 
meni  C  orporation  time-sharing  systems  in  Ihe  proceedings  of  ihe 
Second  Congress  on  Ihe  Information  Syslem  Sciences  (Spiegel 
and  Walker,  1965). 
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capability  will  force  the  planner  to  think  through  his 
problem  and  to  address  the  broad  issues  — what  as¬ 
sumptions  he  makes,  what  options  he  explores,  what 
criteria  he  uses  — instead  of  being  concerned  almost 
exclusively  with  clerical  manipulations. 

A  flexible,  on-line  computer  system  for  using  a 
variety  of  models  is  feasible  and  practical. 

V,  Characteristics  of  a  Strategic 
Mobility  Analysis  Capability 

A.  Variety  of  analysis  tasks 

The  discussion  of  the  strategic  mobility  problem 
indicated  that  there  is  not  just  one  “strategic  mobility 
problem,”  but  that  there  are  many  different  problem 
formulations  of  interest.  The  basic  issue  in  strategic 
mobility  is  the  balancing  of  resources  against  re¬ 
quirements  with  respect  to  various  performance 
criteria.  There  are  a  wide  variety  of  different  types  of 
transportation  resources,  including  vehicles  and  net¬ 
works.  There  is  not  one  single  criterion  for  effective¬ 
ness  of  a  mobility  plan  applicable  in  all  contexts, 
but  a  variety  of  potentially  significant  measures  of 
effectiveness  must  be  considered.  The  strategic 
mobility  triangle  focuses  on  the  variety  of  problem 
types  in,  which  different  elements  of  the  triangle  may 
be  stated  as  given,  with  the  remaining  elements  to 
be  found  by  analysis. 

The  core  of  the  strategic  mobility  triangle  is  move¬ 
ment  routing  and  scheduling.  Even  when  the  type  of 
analysis  does  not  require  the  level  of  detail  repre¬ 
sented  by  a  complete  movement  schedule,  routing 
and  scheduling  issues  are  still  involved.  This  is  partic¬ 
ularly  important  in  understanding  the  relationships 
among  different  types  of  transportation  models. 
Typically,  strategic  planning  problems  are  divided 
by  time  frames  into  long-range,  mid-range,  con¬ 
tingency  and  operational  planning,  and  current  opera¬ 
tions.  The  triangle  of  resources  — requirements  — 
criteria  is  valid  for  each  time  frame.  Other  distinctions 
of  problem  types  can  be  made,  using  the  triangle  as 
a  base,  and  corresponding  to  analysis  functions. 

The  discussion  of  strategic  mobility  problems 
indicates  conclusively  that  there  are  in  fact  a  variety 
of  problem  types. 

B .  Variety  of  models 

Next,  we  discussed  models  required  to  do  strategic 
mobility  analysis.  We  enumerated  basic  principles  of 
transport  systems  analysis  and  showed  their  appli¬ 
cation  to  strategic  mobility. 

These  principles  had  a  number  of  specific  impli¬ 
cations  for  modeling  strategic  transportation.  In 
particular,  we  discussed  in  detail  the  relationships 
between  alternative  levels  of  strategic  transportation 


models,  ranging  from  detailed  vehicle  and  network 
models  to  gross  flow  models.  We  identified  possible 
relationships  between  these  models.  The  basic  and 
most  important  conclusion  of  this  argument  is  that 
there  is  no  single  model  which  is  the  model  to  use  in 
all  strategic  mobility  analyses. 

C.  Systems  analysis  as  a  framework 

The  basic  theory  of  what  has  come  to  be  known  as 
systems  analysis  was  discussed  next.  The  sequence 
of  analysis  implied  by  this  theory  was  shown  to  be 
an  ideal  which  is  rarely  achieved  in  practice.  A  more 
realistic  sequential  approach  was  identified  which 
preserved  the  essential  characteristics  of  systems 
analysis,  while  recognizing  the  realities  of  limitations 
of  mobility  analysis  resources  and  tools.  The  systems 
analysis  ideal  provides  a  useful  objective  for  design 
of  an  analysis  capability,  but  the  analysis  capability 
must  recognize  that  the  ideal  will  not  be  achievable 
in  practice.  A  variety  of  capabilities  are  identified  to 
provide  the  analyst  the  capability  for  striving  toward 
the  systems  analysis  ideal  as  effectively  as  possible. 
Essentially,  in  conjunction  with  the  recognition  of  a 
variety  of  models  for  strategic  mobility  analysis, 
the  objective  of  systems  analysis  argues  for  a  highly 
flexible  analysis  capability,  in  which  the  planner  can 
modify  his  analysis  in  many  ways  as  he  explores  the 
scope  of  his  problem. 

D.  Flexible  analysis  capability 

Discussion  of  the  analysis  environment  began  with 
recognition  of  the  variety  of  modes  of  utilization  of  an 
analysis  capability  by  the  analyst.  The  basic  tech¬ 
nological  capabilities  which  can  be  provided  in  a 
highly  interactive  system  were  identified.  Essentially, 
the  idea  of  multi-user,  remote-access,  “time-sharing” 
systems  is  that  they  provide  a  large  number  of 
individuals  access  from  their  desks  to  powerful, 
sophisticated  computing  capabilities.  This  environ¬ 
ment  will  allow  the  planner  to  focus  on  understanding 
a  particular  mobility  problem  and  exploring  the 
issues  of  that  problem,  freeing  him  from  burdensome 
clerical  tasks  and  allowing  him  to  exercise  to  the 
maximum  extent  possible  his  experience  and  judg¬ 
ment. 

E.  Conclusions 

We  conclude  that  a  strategic  mobility  analysis 
capability  must  be  capable  of  addressing  a  variety 
of  analysis  tasks,  must  have  a  variety  of  models 
available  to  be  applied  to  the  tasks  as  the  analyst 
chooses,  must  provide  appropriate  tools  for  organizing 
an  analysis  within  the  systems  analysis  framework, 
and  must  be  accessible  to  the  planner  in  an  inter- 
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active,  responsive,  flexible  way  through  multi-user, 
on-line,  remote-access  computing  capability. 

These  implications  identify  a  number  of  major 
tasks  areas  to  be  addressed  in  constructing  such  a 
capability.  First,  and  most  obviously,  the  initial  out¬ 
lines  of  the  required  strategic  mobility  models  must 
be  established,  and  appropriate  data  collection  and 
model  verification  initiated.  Second,  priorities  must 
be  established  and  models  implemented.  Third, 
analysts  with  extensive  military  experience  and/or 
extensive  systems  analysis  experience  must  be 
brought  into  the  system  development  process  to 
assure  development  of  appropriate  models,  and  to 
be  able  to  make  effective  use  of  those  models.  Fourth, 
the  appropriate  type  of  computer  environment  must 
be  implemented  as  rapidly  as  possible.  Fifth,  basic 
research  must  be  initiated:  to  develop  fundamental 
understanding  of  strategic  mobility  as  a  function,  to 
develop  new  kinds  of  strategic  mobility  models  and 
analysis  techniques,  and  to  develop  theoretical  and 
practical  understanding  of  the  effective  use  of  a  com¬ 
prehensive  and  flexible  analysis  capability. 

F.  Closing  comment 

The  new  user  of  the  computer,  as  well  as  the 
neophyte  systems  analyst,  tends  to  have  an  over- 
simple  view  of  how  a  computer  is  used  in  analysis. 
In  this  view,  the  sequence  of  operations  is  linear: 
assemble  the  data,  feed  the  data  into  the  program, 
and  summarize  the  computer  output. 

In  reality,  analysis  of  problems  as  complex  as  in 
strategic  mobility  involves  many  different  sequences 
of  analysis,  and  many  different  uses  of  the  computer. 
Problem  analysis  is  a  process  — a  process  in  which 
additional  information  is  acquired  as  gaps  and  errors 
are  discovered  in  the  data  base,  a  process  in  which 
models  are  revalidated  and  revised,  and  a  process  in 
which  the  analyst  is  continually  learning  more  about 
the  shape  of  the  issues. 

Strategic  mobility  analysis  must  be  seen  as  a 
process.  The  design  of  an  analysis  capability  must 
address  this  explicitly. 
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Introductory  Remarks 

Norman  Wars 
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Three  design  dimensions 

The  designer  of  a  military  information  system  has 
at  least  three  dimensions  to  his  job  whose  polarities 
he  should  consciously  appreciate.  The  purpose  of 
this  3rd  Congress  session  of  the  panel  is  to  supple¬ 
ment  the  material  provided  by  a  similar  panel  at  the 
2nd  Congress  by  illustrating  and  illuminating  these 
dimensions.  Each  of  the  three  dimensions  will  be 
discussed  in  turn.  Each  one,  in  a  sense,  is  embrasive 
of  those  that  follow  it. 

The  first  dimension 

The  first  and  most  basic  of  these  dimensions  is  the 
intended  intelligence  content  of  the  information  sys¬ 
tem  being  designed.  The  “intelligence”  dimension  is 
meant  here  to  be  that  dimension  along  which  the 
degree  to  which  human  beings  are  informed,  or  pro¬ 
vided  with  knowledge/intelligence,  by  the  system  is 
measured,  The  scale  involved  is  a  function  of  human 
utility,  and  may  even  be  negative  (i.e.,  have  dis-utility 
to  people).  Thus  we  may  distinguish  between  those 
information  systems  on  one  end  of  the  spectrum  of 
the  general  class  of  information  systems  which  have 
little  direct  human  value  and  the  others.  The  former 
might  be  thought  of  simply  as  “data”  systems,  or 
systems  in  which  the  intelligence  content  of  what  is 
being  transmitted  is  neutral  or  low  in  human  terms, 
problems  for  their  designers.  Eldridge's  paper  is, 
in  fact,  an  example  of  the  serious  problems  one  can 


run  into  in  designing  such  a  system.  In  particular, 
he  attempts  to  illustrate  the  difficulties  one  encounters 
in  performing  the  systems  analysis  related  to  such 
design  effort. 

Carroll  and  Zannetos  on  the  other  hand,  attempt  to 
show  some  of  the  difficulties  of  designing  what  might 
be  classified  as  “highly  intelligent”  information  sys¬ 
tems.  They  also  try  to  show  conceptually  at  what  point 
in  the  spectrum  of  information  systems  such  a  system 
may  first  be  classified  as  an  “intelligent”  system  in 
the  first  place. 

The  second  dimension 

A  second  scale  along  which  a  military  information 
system  designer  might  usefully  classify  his  efforts  in 
an  explicit  way  is  in  terms  of  available  conceptual  and 
technological  state-of-the-art.  For  military  information 
systems  on  one  end  of  the  spectrum,  like  weapon  or 
other  vehicle  control  systems,  the  concepts  and  the 
technology  may  be  considered  to  be  available,  for 
all  practical  purposes.  What  needs  to  be  appreciated 
here  is  only  the  massive  systems  engineering  effort 
that  must  be  organized  to  acquire  such  systems.* 

On  the  other  end  of  the  spectrum,  however,  are 
the  so-called  “command”  or  “command  control” 
types  of  military  information  systems.  Here  we  have 
a  type  of  system  whose  available  conceptual  and  tech¬ 
nological  inventory  is  low,  undoubtedly  because  they 
so  involve  man  and  his  higher  order  mental  processes 
as  a  basic  part  of  the  system.  Significantly,  Baruch’s 

*See  D.  Israel’s  paper  “System  Engineering  Experience  with 
Automated  Command  &  Control  Systems,”  elsewhere  in  this 
volume,  for  a  detailed  treatment  of  the  needed  systems  engineering 
effort. 
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paper,  on  an  entirely  different  field  than  the  military 
—  hospital  information  systems  — amply  illustrates 
that  the  problem  is  not  peculiar  to  military  infor¬ 
mation  systems  at  this  end  of  the  spectrum.  Indeed, 
as  he  and  the  literature  illustrate,  military  information 
systems  of  this  type  share  quite  common  needs  for 
concepts  and  technology  with  “information”  systems 
being  provided  to  serve  a  number  of  other  environ¬ 
ments  or  professions  like  medicine,  law,  and  educa¬ 
tion. 

The  third  dimension 

This  last  leads  to  the  third  dimension  by  which 
information  systems  as  a  class  might  be  measured, 
with  consequent  significance  to  designers.  And  that 
is  the  purpose,  in  professional  or  management  terms, 
intended  to  be  accomplished  by  the  system.  Clearly 
there  is  a  difference  between  the  approach  that  must 
be  taken  to  designing  a  management  information 
system  for  strategic  planning  purposes  and  that 
which  is  being  provided  for  various  types  of  control 
purposes. 

McKenney’s  paper  is  a  detailed  case  example  of  an 
attempt  to  create  the  former  type  of  system,  focusing 
on  the  simulation  model  involved.  The  Carroll/ 
Zannetos  paper  develops  some  of  the  designer’s 
concerns  when  attempting  to  provide  “operating 
process  control”  and  “planning  process  control” 
information  systems. 

SUMMARY  CONCLUSIONS 

By  way  of  introduction  and  synthesis  of  the  indi¬ 
vidual  papers  that  follow,  selected  basic  conclusions 
reached  by  each  of  their  authors  are  set  down  here 
for  readers’  comparison.  No  attempt  has  been  made 
to  edit  these  conclusions,  except  to  list  them  in 
descending  order  of  specificity  of  the  problem  they 
address:  from  the  very  general  to  the  very  pointed. 

“We  conclude  from  our  brief  review  that  there 
exists  no  publicized  comprehensive  realization  of 
intelligent  management  information  systems.” 

Carroll  and  Zannetos 


“/ n  total,  many  of  the  bits  and  pieces  from  which 
higher  level  intelligence  in  management  infor¬ 
mation  systems  can  be  fabricated  exist.  The  prob¬ 
lem  is  to  assemble  these  within  one  organization.” 

Carroll  and  Zannetos 


“ In  conclusion ,  we  believe  that  increased  orga¬ 
nizational  intelligence  is  possible  and  greatly  to 


be  facilitated  by  new  advances  in  information 
technology.  Perhaps  the  greatest  progress  can  be 
made ,  however,  by  simply  recognizing  that  in¬ 
creasing  intelligence  is  a  legitimate  goal  for  in¬ 
formation  systems  design ,  and  that  there  are 
some  straightforward  steps  which  can  be  taken 
towards  that  goal.” 

Carroll  and  Zannetos 


“ The  professional-level  dedicated  system  differs 
both  in  degree  and  kind  from  the  general  system. 
The  involvement  in  the  users  problem  it  demands 
of  the  system  designer  represents  both  a  chal¬ 
lenge  and  an  opportunity.  It  is  our  belief  that 
there  are  so  tnany  professional  areas  that  need 
such  help :  education ,  the  law ,  architecture, 
libraries,  etc.,  that  the  more  generalization  we 
can  do  from  our  specific  observations  the  better 
off  all  of  society  will  be.  What  is  obvious  in  one 
specialty  may  well  not  be  in  another.” 

Baruch 


“While  our  past  five  years  of  experience  have 
been  associated  with  one  hospital  within  the 
medical  profession ,  we  believe  that  the  four  out¬ 
standing  requirements  of  which  we  have  become 
aware  are  gerieralizable  to  other  professional 
level  dedicated  systems.  We  have  found  that 
our  system  must  be: 

1 .  Responsive 

2.  Reliable 

3.  Unobtrusive 

4.  Modifiable” 

Baruch 


“ Better  models  are  the  result  of  a  joint  develop¬ 
ment  effort  of  the  individuals  who  are  to  use  the 
model  as  a  tool  (the  planners)  and  the  individuals 
who  are  designing  and  programming  it  (the 
modelers).” 

McKenney 


”The  most  appropriate  .  . .  reference  base  .  .  . 
for  the  initial  stages  of  model  development ...  is 
the  planners 9  definition  of  the  pertinent  in¬ 
fluences  in  the  environment  and  how  he  thinks 
they  relate  to  each  other.” 

McKenney 
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" This  improvement  (in  the  modeling)  process 
seems  to  he  one  of  continuous  rede  finition  of  the 
panners *  concept  of  the  pertinent  forces  in  the 
environment  and  growth  in  the  modelers*  ability 
to  adequately  represent  these  forces .'* 

Me  Kenney 


“An  impediment  to  more  adequate  rapport  be¬ 
tween  modeler  and  planner  is  the  state  of  present 
computer  languages.  The  language  of  the  pro¬ 
gram  for  the  mode!  has  to  be  interpreted  to  the 
planner.  This  interpretation  creates  ambiguities 
and  misunderstandings  which  limit  the  effec¬ 
tiveness  of  present  simulations  as  a  tool  for  most 
planners.  Hopefully  new  computer  languages 
will  allow  the  modeler  and  planner  to  concep¬ 
tualize  the  simulation  mode!  in  the  language  it  is 
to  be  programmed.** 

McKenney 


4 7  will  define  information  systems ,  here ,  as  any 
network  of  facilities  that  acquires  and/or  pro¬ 
cesses  data  for  use  in  controlling  resources. 
In  this  sense  an  information  system  can  be  either 
purely  automatic  or  it  can  contain  manual  ele¬ 
ments  for  monitoring ,  display ,  control ,  or  other 
command  functions. 

“ Associated  with  every  decision  are  both  pon¬ 
derable  issues  and  imponderable  ones.  The  an¬ 
alyst  should  concentrate  on  the  ponderable  ones. 
The  decision  maker  must  consider  both. 

44 There  are  several  alternative  approaches  open 
to  the  systems  analyst  who ,  once  having  identi¬ 
fied  the  principal  issues ,  undertakes  to  address 
them.  He  can  collect  and  analyze  raw  data  or  he 


can  build  models  of  the  system  and  synthesize 
data  that  will  serve  to  answer  the  questions  or  he 
can  do  both.  The  analyst  is  advised  to  look  at 
both  the  hardware  and  the  software,  or  (the) 
operational  aspects ,  of  the  system,  in  order  to 
develop  a  balanced  context  for  his  study.  He 
should  relate  the  objectives  and  the  missions 
of  a  support  system,  such  as  telecommunications , 
to  the  objectives  and  missions  of  the  system  that 
it  supports. 

44 The  choice  of  suitable  measures  of  effective¬ 
ness  is  a  critical  part  of  the  analysis.  He  should 
be  aware  of  what  the  system  can  do  as  well  as 
what  it  should  do.  By  choosing  measures  of 
effectiveness  that  are  related  to  the  missions  of 
the  system  supported  he  can  avoid  selecting 
effectiveness  criteria  that  relate  to  less  impor¬ 
tant  side  issues. 

“ Another  critical  function  of  the  analyst  is  the 
definition,  for  the  decision  maker,  of  suitable 
alternative  systems  and  methods  of  operation. 
The  analyst  should  consider  a  large  enough  study 
context  to  allow  for  development  of  feasible  al¬ 
ternatives  and  trade-offs  and  to  avoid  unsuit¬ 
able  sub-optimal  decisions.  For  instance,  dif¬ 
ferent  time  phasings  of  any  one  program  are 
important  alternatives  to  be  considered.  How¬ 
ever,  the  cost  and  effectiveness  of  alternative 
programs  are  also  usually  very  relevant. 

“The  analyst  should  call  the  decision  maker's 
attention  to  uncertainties  in  his  study  and  should , 
where  possible,  provide  him  with  sensitivity 
analyses  for  important  but  uncertain  phasing, 
effectiveness  and  cost  factors. 

“ Finally ,  he  should ,  where  possible,  indicate 
which  factors  cannot  be  analyzed  quantitatively 
within  the  time  frame  of  the  study  or  the  context 
of  the  data  available  and,  therefore,  should  be 
left  to  the  judgment  of  the  decision  maker." 

Eld  ridge 
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INTRODUCTION 

Since  1962  an  intensive  research  effort  has  been  under 
way  on  a  collaborative  basis  between  computer  tech¬ 
nologists  at  Bolt  Beranek  and  Newman  Inc.  and  hos¬ 
pital  personnel  at  the  Massachusetts  General  Hospital. 
Under  this  project,  a  PDP-1  digital  computer  has  been 
operated  as  a  dedicated  system  under  a  Time-Sharing 
Executive  to  provide  multiple  terminal  access  at  sev¬ 
eral  experimental  locations  in  the  hospital.  During  these 
past  four  years,  various  hospital  personnel  with  a  wide 
range  of  training  have  operated  the  terminals  experi¬ 
mentally  in  such  applications  as  admissions,  bed  oc¬ 
cupancy,  the  ordering,  administration  and  control  of 
medications  and  laboratory  test  reporting. 

A  large  library  of  programs  which  fluctuates  in  size 
and  composition  is  on  easy  call  from  the  hospital  ter¬ 
minals  and  permits  the  various  research  teams  under 
the  direction  of  Dr.  G.  Octo  Barnett  to  experiment  with 
and  evaluate  the  use  of  the  terminals,  programs  and  the 
computer  lying  behind  them  in  typical  hospital  pro¬ 
cedures. 

The  computer  equipment  is  located  approximately 
seven  miles  from  the  hospital  and  all  connections  to  it 
are  made  by  telegraph-grade  land  lines.  The  types  of 
programs  implemented,  the  equipment  used  for  their 
implementation  and  much  of  the  hospital  reaction  has 
already  been  reported  either  in  the  professional  litera¬ 
ture  or  in  technical  reports  of  the  project,  a  list  of 
which  is  included  at  the  end  of  this  paper.  The  four 
years  have,  however,  made  clear  the  existence  of  certain 
needs  and  desiderata  which  a  medical  computer  center 
must  fill  if  it  is  to  serve  as  a  useful  dedicated  system. 
It  is  hoped  that  the  nature  of  these  needs  and  some  of 
the  steps  presently  being  taken  to  meet  those  needs  on 
a  practical  scale  may  be  of  use  to  those  readers  who  arc 

♦The  work  reported  here  was  done  in  part  while  the  author 
was  at  Bolt  Beranek  and  Newman  Inc.,  Cambridge,  Massa¬ 
chusetts.  That  work  was  performed  under  Contract  PH43-62- 
850,  and  Grant  £GM  00263-0!  with  the  National  Institutes 
of  Health,  U.S.P.H.S.  and  under  a  grant  from  the  American 
Hospital  Association. 


concerned  with  the  implementation  of  other  similar 
systems. 

A  matter  of  definition 

In  the  preceding  seetion  we  used  the  words  “dedi¬ 
cated  system.”  Sueh  a  system,  in  our  case,  is  dedicated 
to  the  service  of  professionals  engaged  in  the  practice  of 
their  procession.  As  sueh,  it  has  many  of  the  character¬ 
istics  of  an  information  system  used  for  military  com¬ 
mand  and  control,  for  legal  analysis  and  research  or  for 
design.  A  dedicated  system  has  certain  characteristics 
which  distinguish  it  from  the  general  purpose  informa¬ 
tion  system. 

Some  of  the  distinguishing  characteristics  arc: 

•  Users  of  the  dedicated  system  may  generally  be 
assumed  to  be  unfamiliar  with — and  indifferent 
to — the  operation  of  a  computer. 

•  While  the  general  purpose  system  concentrates 
on  supplying  computational  power,  the  dedicated 
system  serves  more  as  a  data  repository-com¬ 
munication  system.  As  a  result,  data  input,  re¬ 
trieval,  description  and  dissemination  become 
major  system  functions  with  secondary  attention 
paid  to  raw  computational  power. 

•  The  dedicated  system  serves  a  reasonably  well- 
defined  population  dealing  with  a  generally  re¬ 
stricted  vocabulary.  Further,  the  problem  areas 
are  relatively  circumscribed  at  the  initial  stages 
of  a  dedicated  system. 

While  these  distinctions  may,  at  first  blush,  appear 
to  be  designers’  advantages,  in  fact  they  imply  certain 
responsibilities  if  the  long-term  needs  of  the  user  group 
are  to  be  met. 

In  an  effort  to  generalize  from  the  specific  case  of  the 
medical  computer  system,  let  us  examine  some  of  the 
needs  which,  to  us,  seem  to  be  characteristic  of  a  dedi¬ 
cated  system  when  used  at  the  professional  level.  These 
needs  may  or  may  not  be  self-evident.  To  us — at  the 
start — their  importance  was  not  fully  appreciated.  They 
have,  however,  become  the  most  important  guides  we 
have  in  the  second-pass  system  now  under  construction. 
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Needs  and  desiderata 

While  our  past  five  years  of  experience  have  been 
associated  with  one  hospital  within  the  medical  profes¬ 
sion,  we  believe  that  the  four  outstanding  requirements 
of  which  we  have  become  aware  are  generalizable  to 
other  professional — level  dedicated  systems.  We  have 
found  that  our  system  must  be: 

1  Responsive 

2  Reliable 

3  Unobstrusive 

4  Modifiable 

Let  us  examine  each  of  these  needs  in  some  detail. 
In  our  section  on  current  approach  we  will  discuss  how 
to  meet  them. 

Responsiveness 

A  professional  using  an  information  manipulation 
system  is  generally  using  it  as  an  adjunct  to  his  own 
thoughts  and  decision-making  processes.  As  such,  it  is 
important  that  there  be  a  reasonable  match  between 
the  system  time  constant  and  the  time  constants  of  the 
individual  engaged  in  these  processes.  Further,  the  user’s 
satisfaction  with  the  responsiveness  of  the  system  is 
often  conditioned  by  his  expectations.  These  expecta¬ 
tions  are  based  on  his  past  experience  and  his  estimate 
of  the  “work”  involved  in  what  the  machine  is  doing. 
These  two  aspects  control  responsiveness-in-time. 

A  simple  example  will  illustrate  time  responsiveness. 
The  nurse  or  physician  engaged  in  a  patient  care  situa¬ 
tion  requires  answers  in  a  time  measured  in  seconds; 
the  researcher  engaged  in  file  retrieval  exercises  re¬ 
quires  his  answers  in  minutes  or  hours.  Systems  designed 
to  respond  to  the  researcher  much  faster  than  this 
represent  a  wasted  asset.  The  “mulling  time”  required 
in  his  overall  task  is  such  that  the  total  elapsed  time 
from  start  of  a  problem  solution  to  its  end  will  not  be 
markedly  decreased  by  increasing  the  response  speed 
of  the  information  processing  system  alone.  If,  on  the 
other  hand,  the  response  time  is  much  slower  than  the 
individual’s  “mulling  time”  wc  have  the  situation  of 
the  individual  waiting  for  the  machine,  losing  his  train 
of  thought,  losing  interest  in  the  problem  or  bypassing 
the  direct  inquiry  use  of  the  on-line  system. 

This  avoidance  of  direct  inquiry  is  most  clearly 
demonstrated  in  the  ordinary  batch  processing  system. 
Here  the  user  commonly  asks  for  many,  many  tables  at 
any  one  run.  He  then  modifies  his  inquiry  behavior 
from  inquiring  of  the  data  to  inquiring  of  this  mass  of 
tables.  Time  responsiveness  avoids  this  form  of  be¬ 
havior. 

It  is  also  interesting  to  note  that  if  a  user  has  grown 
accustomed  to  half-second  response  in  an  operation  he 
does  frequently,  then  an  occasional  three-second  re¬ 


sponse  is  virtually  intolerable.  We  believe,  although  we 
cannot  yet  prove  it,  that  if  the  three-second  response 
had  been  there  initially,  no  adverse  reaction  would 
have  been  noted. 

A  further  characteristic  of  a  responsive  system  is  its 
participation  in  the  activity  and  education  of  the  user. 
Responsive  in  time,  the  system  must  also  be  responsive 
in  kind.  Physicians,  nurses  and  researchers  must  all  be 
assumed  to  be  untrained  in  the  use  of  sophisticated 
informational  systems;  the  computer  must,  therefore, 
be  programmed  to  participate  in  their  guidance.  A 
responsive  system  makes  the  entry  of  data  and  the 
phrasing  of  analysis  requests  a  simple  and  nearly  fool¬ 
proof  procedure.  Further,  error  correction,  procedural 
inquiry,  and  mind-ehanging  are  all  to  be  facilitated  in 
a  truly  responsive  system. 

Consider  the  data  input  or  capture  problem.  We  can 
start  off  by  saying  that  the  general  format  will  be  a 
question-answer  program,  with  the  machine  asking  a 
question  and  the  user  supplying  an  answer.  Such  a 
technique  is  analagous  to  the  pre -printed  form.  For 
a  system  to  be  responsive,  however,  we  require  that 
it  be  better  than  a  form.  Thus,  for  example,  wc  will 
only  ask  those  questions  that  have  a  high  probability  of 
being  pertinent.  In  a  simple  “name-address-phone  num¬ 
ber”  program  we  will  ask  all  the  questions,  whereas  in 
a  more  complex  program  (a  situation  assessment,  for 
example)  we  may  only  ask  a  small  percentage  of  the 
total. 

Further  to  its  responsiveness,  the  system  must  be 
able  to  answer  questions  about  itself  and  its  expecta¬ 
tions  during  the  above  procedure.  For  example,  what 
format  is  required  for  this  answer,  what  did  I  answer 
earlier,  etc. 

Lastly,  in  our  example,  the  system  must  provide  for 
the  frailty  of  the  human  operator.  The  user  must  be 
able  to  correct  his  mistakes  as  he  goes  along.  In  gen¬ 
eral,  these  corrections  will  imply  new  questions  to  be 
answered  and  new  places  for  errors  to  occur. 

While  we  have  here  considered  only  data  input, 
responsiveness  in  kind  must  also  be  evident  in  the  in¬ 
quiry  programs.  More  will  be  said  about  this,  however, 
under  modifiability. 

Reliable 

Outside  of  the  military  sphere,  the  medical  com¬ 
munity  probably  has  the  highest  generally  accepted 
requirement  on  system  reliability  of  any  group  of  users. 
Failure  in  a  hospital  or  other  medical  setting  is  con¬ 
stantly  guarded  against  and  every  effort  is  taken  to 
ensure  an  ongoing  reliable  environment.  The  introduc¬ 
tion  of  automated  information  handling  must  in  no  way 
reduce  the  reliability  of  that  environment. 

It  is  interesting  to  note  that  operating  the  computer 
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system  in  parallel  with  the  older  manual  systems  during 
early  experimental  phases  seems  to  be  a  snare  and  a 
delusion.  In  a  dedicated  system,  which  generally  implies 
a  stressed  system,  the  introduction  of  a  simpler  tech¬ 
nique  in  parallel  with  the  standard  techniques  rapidly 
results  in  the  standard  techniques  being  ignored  and 
atrophying.  Cautions  about  the  possibility  of  failure  go 
unheeded  if  the  failure  probability  is  reasonably  low.  If 
the  failure  probability  is  high,  the  system  will  not  even 
be  amenable  to  valid  experimentation.  We  have  learned, 
therefore,  that  reliability  must  be  built  in  even  at  the 
early  experimental  system  level  and  that,  in  an  un¬ 
structured  community,  one  cannot  rely  on  parallel 
operation  as  a  safeguard  in  the  early  stages  of  activity. 

From  our  experience  we  have  concluded  that  sys¬ 
tem  reliability  in  the  medical  community  must  provide 
for  several  levels  of  failure,  leading  to  the  term  “fail- 
soft”  rather  than  “fail-safe.”  We  recognize  that  the 
level  of  reliability  provided  represents  a  balance  between 
the  cost  of  failure  and  the  cost  of  providing  that  level 
of  reliability.  In  the  medical  as  in  the  military  profes¬ 
sions,  it  is  generally  terribly  difficult  to  assess  the  cost 
of  failure;  these  decisions  are  thus  generally  made 
heuristically  at  the  command  level. 

One  special  word  about  reliability  must  be  entered 
here  and  that  is  the  reliability  and  preservation  of  the 
data  base  itself.  The  professional-level  dedicated  sys¬ 
tem  has,  as  a  major  advantage,  the  provision  of  a  com¬ 
munal  shared  data-base.  Much  of  its  strength  comes 
from  that  common  store  of  information.  Representing 
a  major  advantage,  that  data  base  also  represents  a 
major  responsibility.  One  finds  that  it  is  most  essential 
to  preserve  the  data  base  regardless  of  the  kinds  of 
failure  that  may  take  place  within  the  system  hardware 
or  software.  Indeed,  I  estimate  that  the  social  impor¬ 
tance  of  such  data  bases  will  eventually  lead  to  major 
legislation  in  this  area. 

Unobstrusive 

A  profession  engaged  in  the  practice  of  its  profes¬ 
sion  looks  for  aid  in  an  existing  environmental  context. 
Having  evolved  over  the  years,  there  is  a  high  probabil¬ 
ity  that  the  operating  system  of  that  profession  is  rea¬ 
sonably  efficient  if  not  optimum,  but  lags  its  technology. 
The  introduction  of  a  major  technological  change  such 
as  an  information-handling  system  will  imply  a  poten¬ 
tial  for  change  in  some  of  the  operating  characteristics 
of  the  profession.  The  discontinuity  caused  by  its  in¬ 
troduction  must,  however,  be  minimized  if  the  system 
is  to  be  both  effective  and  accepted.  We  have  found 
it  to  be  essential  that  the  newly  introduced  system  im¬ 
pose  no  requirement  for  the  reorganization  of  the  op¬ 
erational  environment.  The  new  system  must  appear  to 


be  a  natural  extension  to  already  existing  information 
handling  functions. 

Ideally,  an  unobstrusive  information  system  is  one 
that  produces  no  marked  change  in  immediate  function 
but  a  marked  change  in  both  ultimate  capability  and 
the  rate  at  which  the  operating  environment  approaches 
that  capability.  It  is  our  current  hypothesis  that  users 
are  sensitive  to  changes  in  form  but  not  to  changes  in 
the  first  time  derivative  of  form. 

Modifiable 

It  is  our  belief  that  the  most  important  single  char¬ 
acteristic  of  an  information-handling  system  in  a  pro¬ 
fession  is  its  modifiability  by  the  practitioners  of  that 
profession.  It  is  our  major  thesis  that  the  introduction  of 
an  information-handling  system  in  a  profession  can  in¬ 
crease  that  profession’s  capability  for  growth  tremen¬ 
dously.  As  a  corollary,  wc  believe  that  the  introduction 
of  a  rigid  information-handling  system  can  cripple  that 
growth,  making  the  profession’s  potential  completely 
unrealizable.  Casting  the  information  patterns  of  a  pro¬ 
fession  in  concrete,  providing  rigid  “hard  wired”  com¬ 
munication  paths  or  trying  to  anticipate  all  of  the  in¬ 
formation  requirements  for  the  profession  represent 
tremendous  dangers.  We  consider  that  the  information 
environment  must  provide  a  nurturing  effect  and  hence, 
must  facilitate  change  rather  than  restrict  it. 

In  reviewing  the  required  kinds  of  modification,  it 
has  become  clear  to  us  that  modifications  should  not  be 
interccssionist.  One  docs  not  wish  to  create  a  “priest¬ 
hood”  of  programmers  or  the  ilk  through  which  all 
changes  in  the  information-handling  of  the  profession 
must  pass.  Rather,  we  would  like  the  practitioner  him¬ 
self  to  be  able  to  experiment  with  the  configuration  of 
his  environment,  to  change  the  kinds  of  information  it 
contains,  the  definitions  and  the  flow  patterns  of  that 
information.  By  providing  such  experimental  capability 
with  minimal  learning  on  the  part  of  the  practitioner, 
we  will  encourage  the  user  to  recognize  new  needs  and 
try  to  develop  new  scopes  for  his  dedicated  system. 

Current  approach 

The  MEDINET  Department  of  General  Electric 
Company  is  currently  engaged  in  the  design  and  con¬ 
struction  of  a  large  system  for  use  by  the  medical  com¬ 
munity.  The  new  system  aims  at  serving  many  organiza¬ 
tions  that  are  widely  diverse.  Let  us  examine  that  system 
for  the  design  approaches  being  taken  to  meet  the  four 
needs  that  we  have  just  discussed. 

Responsiveness 

The  current  system  is  being  designed  around  a  rela¬ 
tively  standard  Time-Shared  digital  computer  with  three 
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memory  elements  in  a  hierarchical  structure.  As  is  the 
ease  with  many  other  such  systems,  core  is  used  for 
running  programs,  a  high-speed  drum  is  used  for  the 
exchange  of  pages  and  a  large  disc  is  used  for  the  bulk 
storage  of  libraries,  data,  dictionaries  and  so  forth. 
Provision  is  made  for  removing  patients  from  active 
status  to  inactive  status  by  transferring  their  records  to 
tape  and  for  making  such  transfers  reversible. 

Each  program  on  the  library  is  to  have  one  or  more 
queuing  codes  incorporated  within  it  to  control  the 
minimum  service  level  of  that  program  during  busy 
running  periods.  The  resulting  priority  chain  will  thus 
be  a  combination  of  first-come,  first-served  modified 
by  minimum  levels  of  serviceability  associated  with 
each  program.  Such  a  system  provides  a  guarantee  that 
no  low-urgency  program  will  detract  from  a  high- 
urgency  program. 

To  provide  for  high-speed  response  during  the  data 
input  phase  where  the  user’s  speed  expectation  is  high, 
peripheral  computers  are  used  for  message  control.  To 
provide  for  stability  of  response  in  order  that  the  user’s 
expectation  not  be  built  up  during  periods  of  low  sys¬ 
tem  use,  an  experimental  initial  artificial  delay  has 
been  built  into  the  system  to  prevent  responses  shorter 
than  2 Vi  seconds.  Early  use  of  the  system  will  indicate 
whether  this  response  time  is  appropriate. 

In  the  area  of  kind-responsiveness,  we  are  funda¬ 
mentally  concerned  with  facile  data  capture.  In  order 
for  the  system  to  be  both  responsive  and  unobstrusivc, 
it  is  necessary  that  the  entry  of  data  be  matched  to  the 
professional  level  of  those  entering  it.  Broadly  speak¬ 
ing,  we  deal  with  two  kinds  of  data  entry,  extracts 
from  an  extensive  vocabulary  and  extracts  from  a  lim¬ 
ited  vocabulary.  In  the  ease  of  the  extensive  vocabulary, 
as  encountered  in  making  comments,  random  notes, 
and  free  text,  we  provide  a  keyboard  entry  device. 
Such  a  device  may  be  used  as  an  adjunct  to  dictation  or 
transcription  or  may  be  used  directly  in  the  operating 
situation. 

By  far  the  greatest  quantity  of  data  input  in  the 
medical  environment,  however,  is  of  the  limited- 
vocabulary  type.  For  this  purpose  a  densely  coded 
entry  device  is  provided  on  each  terminal.  Operating 
in  conjunction  with  tables  stored  in  memory  and  with 
overlays,  a  multi-slide  projector,  and  typescripts,  the 
Dataeoder  permits  entry  of  phrases,  expressions  and 
other  data  macros  at  a  rate  exceeding  one  per  second. 
The  vocabulary  represented  by  either  a  small  collec¬ 
tion  of  overlays  or  a  small  set  of  slides  is  so  very  large 
as  to  permit  major  branching  input  programs  with  very 
few  inapplicable  choices. 

This  kind  of  system  that  replaces  a  check  sheet  or 
printed  form  with  the  presentation  of  alternatives  whose 


pertinence  to  the  situation  is  determined  by  the  pro¬ 
gram  permits  the  logical  entry  of  large  masses  of  data 
in  a  very  short  amount  of  time. 

Reliability 

In  order  to  provide  for  gross  reliability,  the  computer 
center,  designed  to  serve  a  group  of  hospitals  and  other 
users  in  the  medical  community,  will  contain  two  com¬ 
plete  computer  systems.  The  present  system  design, 
being  restricted  to  state-of-the-art  techniques,  calls  for 
reliance  on  users  and  operators  for  malfunction  detec¬ 
tion.  Correction  of  such  malfunction  will  be  made  by 
switching  to  the  standby  system  while  the  malfunction 
is  being  tracked  down.  Although  the  status  of  running 
programs  may  be  lost  at  such  a  time,  they  can  generally 
be  reinstated  by  a  small  amount  of  user  repetition. 

In  the  case  of  hardware  malfunction,  such  a  shut¬ 
down  and  transfer  procedure  will  be  generally  both 
necessary  and  sufficient.  In  the  case  of  a  software  mal¬ 
function,  the  ability  to  resume  operation  while  having 
a  completely  frozen  total  system  available  for  fault 
analysis  makes  the  correction  of  software  problems  a 
great  deal  easier  than  would  be  the  ease  were  only  one 
system  available.  In  order  to  protect  against  failure 
because  of  power  loss,  each  center  is  to  contain  a  two- 
stage  standby  power  system  capable  of  taking  over 
with  zero  loss  of  operating  time  should  commercial 
power  fail. 

Should  any  major  catastrophic  failure  occur  resulting 
in  either  loss  of  communications,  loss  of  hospital  power 
or  failure  of  both  of  the  computers,  we  must  still  ensure 
that  the  medical  community  can  function.  To  provide 
such  assurance  in  the  hospital,  we  rely  on  the  printed 
word.  Hard  copy  is  generated  at  each  operating  ter¬ 
minal  in  sufficient  quantity  and  with  sufficient  time¬ 
liness  to  provide  the  necessary  data  for  patient  care  and 
organizational  operation.  Thus,  for  example,  drug  lists, 
lab  reports  and  other  clock-type  retrieval  programs 
generate  print-outs  rather  than  just  scope  displays. 

Such  means  should  yield  the  desired  fail-soft  opera¬ 
tion.  A  typical  machine  or  program  failure  may  cause 
a  one-to-five  minute  loss  of  service.  A  more  severe 
problem  may  cause  a  gradual  degradation  because  of  a 
need  to  rely  on  print-outs  and  manual  techniques  for  an 
hour  or  more.  At  no  time,  however,  are  we  in  more 
danger  from  information-system  failure  than  from  hos¬ 
pital  failure. 

As  mentioned  earlier,  preservation  of  the  data  base  is 
a  prime  consideration  in  the  system  design.  To  this  end, 
the  information  on  the  master  disc  file  is  duplicated 
elsewhere  at  the  computer  center.  As  each  entry  is 
made  on  the  running  file,  the  location  of  that  entry 
and  its  contents  are  recorded  on  non-reversible  mag- 
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netic  tape.  The  collection  of  such  tapes  from  time  zero 
would  then  permit  a  very  lengthy  reconstitution  of  the 
file  were  it  ever  accidently  wiped  out.  Naturally,  such 
an  update  is  impractical.  At  reasonable  periods,  the 
tapes  are  used  to  update  a  second  or  standby  disc. 
Upon  completion  of  this  update,  the  standby  disc  is 
dumped  onto  tape.  The  updated  standby  disc  and  any 
tapes  subsequent  to  the  last  update  thus  form  a  com¬ 
plete  file.  In  the  case  of  memory  destruet  commands 
from  the  computer,  or  a  degradation  in  the  accuracy 
of  what  is  being  written  that  goes  undetected  for. 
some  time,  the  audit  tapes  can  be  edited  before  up¬ 
dating  the  standby  disc. 

There  are,  of  course,  many  more  reliability  steps 
that  must  be  taken  in  software,  hardware  and  proce¬ 
dures.  The  above  examples  (the  most  interesting  to 
this  writer)  serve,  however,  to  illustrate  their  level  of 
rquirement. 

Unobtrusiveness 

In  order  that  the  system  be  as  unobtrusive  as  pos¬ 
sible  when  first  introduced,  the  initial  programs  being 
written  for  user  hospitals  arc  designed  essentially  to 
implement  those  information-handling  processes  cur¬ 
rently  in  use.  This  decision  causes  great  strain  on  the 
staff  since  it  calls  for  postponing  many  significant  and 
obvious  improvements.  In  order  to  allow  users  to  use 
words  familiar  to  them  rather  than  learn  new  ones,  we 
have  investigated  the  language  differences  among  hos¬ 
pitals.  We  find  that  the  major  differences  among  hos¬ 
pitals  arc  formal  rather  than  substantive  ones.  We  are 
thus  endeavoring  to  include  large  multiply  overlapped 
dictionaries  in  the  system.  These  will  permit  a  high 
degree  of  data  sharing  as  the  use  of  the  system  develops. 

Contrary  to  our  earlier  expectations,  the  reasonable 
introduction  of  a  dedicated  system  into  the  medical 
community  appears  to  be  readily  acceptable  by  the 
practitioners  of  that  profession.  By  making  the  system 
unobtrusive,  we  arc  trying  to  insure  that  “management” 
will  not  use  “the  machine”  as  an  excuse  to  bring  about 
other  changes  that  it  feels  necessary  or  desirable. 
There  seems  no  more  likely  a  way  of  guaranteeing  the 
rejection  of  a  system  than  by  introducing  a  simultaneous 
set  of  procedural  changes  with  the  comment,  “It  has 
to  be  done  this  way  because  of  the  machine.  .  . 
Despite  the  fact  that  the  system  is  being  made  initially 
unobstrusivc,  extensive  literature  is  being  prepared  to 
provide  instruction  for  the  medical  community  in  the 
system  modification.  Knowledge  of  this  modifiability 
stresses  the  system’s  subservient  role  and  makes  it  more 
acceptable  as  well  as  more  useful. 

Modifiability 

In  order  to  permit  the  members  of  the  medical  com¬ 


munity  to  modify  the  programs  that  they  use,  we  need 
to  express  those  programs  and  store  them  in  an  easily 
understandable  language.  The  rules  concerning  who 
may  change  programs — and  what  ones  they  may  change 
— must  be  specified  ex-system.  They  arc  far  too  complex 
in  any  real  situation  to  permit  of  simple  algorithmic 
statement. 

While  it  is  currently  fashionable  to  seek  for  a  “Nat¬ 
ural  English  Language”  form  of  programming,  par¬ 
ticularly  in  inquiry  work,  as  a  general  cure  to  program¬ 
ming  ills,  we  have  taken  a  somewhat  more  realizable 
approach  that  recognizes  the  special  nature  of  our  users. 
The  RAND  Corporation’s  experience  with  the  JOSS 
language  and  BBN’s  experience  with  TELCOMP  a 
JOSS  derivative,  has  led  us  to  tackle  the  design  and 
implementation  of  F1LECOMP,  a  JOSS-Iike  language 
suitable  to  medicine,  incorporating  file  manipulation 
capabilities  and  an  expandable  library  of  system  verbs. 

Every  general  program  on  the  stored  library  will 
be  carried  in  the  FILECOMP  language  and  may  be 
called  in  that  form  for  modification.  Aside  from  the 
portions  of  the  program  that  are  incorporated  in  order 
to  provide  data  security,  the  user  will  be  able  to  control 
format,  program  logic,  branching,  generated  communi¬ 
cations  and  inter-program  communication  to  any  de¬ 
gree  he  may  desire.  He  will,  of  course,  also  be  able  to 
generate  completely  new  programs. 

Since  a  FILECOMP  Interpreter  is  available  during 
such  a  modification,  the  user  may  experiment  until  he 
lias  a  program  running  to  his  liking.  Once  the  user  has 
a  program  that  satisfies  him,  he  will  be  able  to  add  it 
to  the  library.  From  then  on,  he  and  others  who  have 
access  to  that  private  library  may  call  it  in  “run  mode” 
just  as  they  would  call  any  other  library  program. 

Frequently,  programs  thus  modified  may  be  of  more 
general  public  use.  In  such  eases  they  may — with  the 
author’s  approval — be  transferred  from  a  private  library 
to  the  public  library.  Such  a  transfer  will  be  accom¬ 
panied  by  a  rigid  quality  control  procedure  that  includes 
testing,  evaluation  and  documentation.  It  is  our  hypo¬ 
thesis  that  medical  personnel  will  not  only  use  this 
modification  capability,  but  that  they  will  take  ad¬ 
vantage  of  the  wide  public  library  audience  and  use  it 
as  a  form  of  professional  publication.  Needless  to  say, 
the  requirement  for  modifiability  extends  not  only  to 
programs  but  to  the  data,  the  procedural  tables,  dic¬ 
tionaries,  etc.  that  go  to  make  up  the  overall  system. 

CONCLUSION 

While  this  author’s  understanding  of  the  epistemological 
basis  for  system  theory  is  too  limited  either  to  allow 
him  to  generalize  rigorously  from  his  restricted  experi¬ 
ence  or  to  describe  that  restricted  experience  in  scientific 
terms,  it  is  hoped  that  the  four  needs  uncovered  in  our 
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work  in  the  medical  community  (obvious  as  those  needs 
may  be)  and  the  approach  we  are  taking  to  eope  with 
those  needs  may  serve  as  useful  inputs  to  some  readers. 

The  professional-level  dedicated  system  differs  both 
in  degree  and  kind  from  the  general  system.  The  in¬ 
volvement  in  the  user’s  problem  it  demands  of  the  sys¬ 
tem  designer  represents  both  a  challenge  and  an  op¬ 
portunity.  It  is  our  belief  that  there  are  so  many  pro¬ 
fessional  areas  that  need  such  help:  education,  the  law, 
architecture,  libraries,  etc.,  that  the  more  generalization 
we  ean  do  from  our  specific  observations  the  better  off 
all  of  society  will  be.  What  is  obvious  in  one  specialty 
may  well  not  be  in  another. 
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INTRODUCTION 
Statement  of  purpose 

In  reviewing  existing  statements  on  what  a  manage¬ 
ment  information  system  should  provide,  we  have  noted 
a  singular  lack  of  operationally  viable  goals.  “To  pro¬ 
vide  a  basis  for  better  decision  making”  simply  does 
not  provide  a  basis  for  choice  for  the  system  designer. 
It  is  to  help  fill  this  void  that  we  arc  motivated.  Con¬ 
sequently,  our  purpose  is  to  propose  some  new  doc¬ 
trine  for  management  information  system  design,  to 
state  some  explicit  goals  to  be  sought,  and,  in  so  doing, 
to  offer  some  new  perspectives  for  designers. 

What  we  propose  is  an  information  system  which  aids 
in  continually  increasing  management’s  understanding 
of  its  environment  and  in  improving  management’s 
logic  or  rationality  in  dealing  with  that  environment.  In 
short,  we  are  suggesting  information  systems  which 
will  allow  management  organizations  to  exhibit  more 
intelligent  behavior. 

Let  us  stress  that  it  is  the  organization  which  behaves 
intelligently.  We  are  not  proposing  an  artificially  intel¬ 
ligent  machine  system,  but  rather  men  and  machines 
collectively  and  cooperatively  exhibiting  intelligent  be¬ 
havior. 

Structure  of  what  lies  ahead 

In  the  remainder  of  Part  I  we  discourse  on  the  nature 
of  intelligence  and  management  and  define  our  frame¬ 
work  and  terms.  In  Part  II  we  attempt  to  provide 
specifications  for  intelligent  management  information 
systems.  We  review  the  current  state  of  the  art  in  light 
of  these  specifications  in  Part  III,  and  in  Part  IV  we 


propose  steps  and  some  research  to  be  undertaken  to 
fulfill  the  specifications.  This  is  followed  by  a  brief 
summary  and  conclusions. 

On  the  nature  of  intelligence 

To  provide  a  rigorous  definition  of  “intelligent  be¬ 
havior”  is  extraordinarily  difficult,  but  we  do  wish  to 
offer  a  working  definition,  to  clarify  our  use  of  the 
term  as  applied  to  management  information  systems. 
Intelligent  behavior  is  “doing  what  you  do  for  the  right 
reasons,”  roughly  speaking.  According  to  this  definition, 
intelligence  has  two  facets:  first,  understanding  of  cause 
and  effect,  and  second,  logic  or  rationality  in  employ¬ 
ing  that  understanding  .*  Note  that  we  haven’t  said 
“doing  what  is  right”  in  an  ex  post  facto  sense.  Our 
definition  admits  of  intelligent  behavior  in  the  face  of 
uncertainty;  it  requires  neither  omniscience  nor  clair¬ 
voyance,  and  distinguishes  between  the  decision-making 
process  and  its  outcome. 

Let  us  be  more  specific  about  this  concept  of  “un¬ 
derstanding.”  The  better  one  understands  a  phenom- 
enon,  the  more  accurately  he  can  predict  its  behavior. 
And,  for  purposes  of  subsequent  discussion  we  shall 
find  it  convenient  to  associate  understanding  with  a 
predictive  model  of  a  phenomenon.  That  is  to  say,  by 
increasing  managerial  understanding,  we  shall  mean 
improving  the  manager’s  modelt  to  be  a  better  image 
of  the  real  world — his  models,  whether  they  be  ex- 

*“Underslanding"  corresponds  (approximately)  with  the  in- 
ductive  aspect  of  intelligence  and  “logic"  with  the  deductive. 
tWe  are  implying  many  models  in  the  mind  of  a  manager.  At 
any  point  in  lime,  lhe  models  need  not  be  mutually  consistent, 
humans  being  humans. 
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plicit  or  implicit,  being  his  basis  for  prediction.* * 

The  logic-rationality  aspect  of  intelligence  can  be 
described  in  this  way:  given  a  set  of  relationships  and 
facts,  one  can  be  more  or  less  adroit  in  exploiting  this 
knowledge  for  his  own  ends.  There  arc  at  least  two 
types  of  situations  in  which  logical  use  of  knowledge  is 
at  least  as  important  as  knowledge  itself.  One  is  in 
decision  making  under  uncertainty.  Here  statistical  de¬ 
cision  theory  is  now  offering  a  suitable  rationale  (al¬ 
though  in  problems  of  large  size,  the  computational 
side  of  the  logic  becomes  important).  The  second  situa¬ 
tion  is  in  system  management.  The  detailed  interrela¬ 
tionships  among  system  components  may  be  well  un¬ 
derstood  but  overall  system  behavior  is  not,  either  be¬ 
cause  of  dynamic  effects  or  simply  because  of  the 
combinatorial  nature  of  the  full  system.  Forrester  (1962) 
has  offered  an  approach  in  the  former  ease,  and 
simulation  or  heuristic  programming  some  promise  for 
the  latter. 

We  should  also  emphasize  that  intelligence  is  both  a 
relative  and  dynamic  quality  of  persons  (and  organiza¬ 
tions).  We  cannot,  except  arbitrarily,  categorize  be¬ 
havior  as  intelligent  or  unintelligent,  but  we  can  talk 
about  “levels”  of  intelligence.  We  arc  willing  to  admit 
that  von  Neumann  was  smarter  than  we,  individually 
and  collectively  (we  have  some  suspicions  about  so^c 
others  as  well).  Hence,  we  shall  be  concerned  with 
organizations  which  arc  more  or  less  intelligent.  In 
stating  that  intelligence  is  dynamic,  we  simply  mean 
that  one  can  grow  (or  even  regress)  in  the  qualities 
associated  with  intelligent  behavior.  In  other  words, 
one  can  learn. 

In  the  light  of  the  above,  we  can  now  be  somewhat 
more  precise  in  our  goal:  We  want  information  sys¬ 
tems  designed  to  provide  for  more  intelligent  behavior 
over  time. 

On  the  nature  of  management 

Our  assumption  is  this:  A  management  organization 
manipulates  those  resources  under  its  control  to  achieve 
results  in  accord  with  its  objectives;  this  manipulation  is 
based  on  management’s  collective  understanding  of 
causal  relationships  among  resource  inputs,  processes, 
external  environmental  factors  and  outputs. 

In  our  view,  the  main  management  processes 
which  require  intelligence  arc  planning  and  control.* 

Planning,  as  we  construe  it,  subsumes  all  considera¬ 
tion  of  the  future  course  of  events,  whether  the  time 
scale  is  short  or  long  and  whether  the  process  is  formal 

t Pounds  (1965)  has  made  a  strong  case  for  the  universality 
of  models  in  managerial  behavior,  in  “problem  finding”  as  well 
as  in  problem  solving. 

*Our  classifications,  planning  and  control,  are  quite  inclusive. 
They  correspond  roughly  with  Anthony's  (1965). 


or  informal.  Planning  thus  includes  searching  for  future 
alternative  courses  of  action,  selection  of  goals,  specifi¬ 
cation  of  procedures  to  be  followed  or  resources  to  be 
acquired  and  utilized  for  the  achievement  of  the  chosen 
goals. 

We  distinguish  between  operational  planning ,  in 
which  the  emphasis  is  on  what  the  organization  should 
be  doing  in  the  relatively  short  run  as  constrained  by 
the  dominant  characteristics  of  its  current  structure, 
and  strategic  planning ,  wherein  the  emphasis  is  on  pos¬ 
sible  changes  in  the  dominant  characteristics  of  the 
structure  (physical  or  organizational)  or  in  major  goals. 
The  nature  of  the  planning  process  entailed  is  likely  to 
be  quite  different  in  the  two  types,  to  wit:  Operational 
planning  is  typically  a  continuing,  systematic  process 
whereas  strategic  planning  is  more  often  ad  hoc  and 
unstructured;  operational  planning  is  intimately  linked 
with  control,  providing  milestones  or  other  goals  and 
receiving  current  system  status,  whereas  strategic  plan¬ 
ning  is  likely  to  consist  of  one-shot  (“terminal”)  deci¬ 
sions  only  loosely  linked  to  the  formal  control  system. 

Regardless  of  emphasis,  planning  always  involves  a 
model;  the  model  is  explicit  in  many  cases,  but  is  cer¬ 
tainly  implicit  in  any  activity  that  projects  the  future.* 
Control 

The  object  of  control  is  to  obtain  desired  behavior 
or  results  (often  as  set  forth  in  a  plan).  Given  the 
specification  of  the  desired  behavior  or  result  in  the 
form  of  a  quantified  standard,  goal  or  budget,  there 
are  certain  processes  which  go  into  control.  The  first 
is  measurement  of  the  status  or  performance  of  the 
controlled  entity.  But  the  measurement  has  no  meaning 
until  it  is  juxtaposed  with  the  standard  or  desired  meas¬ 
urement,  consequently  comparison  is  a  second  basic 
process.  The  third  process  is  direction ,  a  process  of  com¬ 
munication  in  which  the  controlled  activity  receives 
signals  to  alter  its  behavior  to  obtain  closer  conformance 
with  that  desired.* 

Measurement,  comparison,  and  direction  are  com¬ 
mon  to  all  “feedback”  control,  but  quite  often  there 
are  additional  processes  present.  Classification  is  one 
such  process.  It  is  required  when  one  measurement 
relates  to  several  different  activities  or  entities  (this  is 
part  of  the  notion  of  “integrated  data  processing”).  The 
fact  of  completion  of  a  task  in  a  manufacturing  shop 
may  be  reflected  in  individual  worker  productivity,  the 
foreman’s  direct  labor  expenditure,  production  progress 
control,  and  the  like.  So  within  the  control  cycle  there 
is  required  association  of  the  measurements  with  the 
appropriate  “responsible”  entities,  which  is  classifica¬ 
tion. 

tFor  a  thorough  discussion  see  Emery  (1965). 

*There  are  situations  in  which  deviations  of  one  direction  only 
are  “bad,”  deviations  of  opposite  sign  therefore  receive  no 
correcting  signal;  if  anything,  they  receive  “reinforcement  ” 
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Also,  implicit  in  comparison-direction  is  the  neces¬ 
sity  for  determining  the  cause  of  deviation  between 
actual  measurement  and  desired.  Diagnosis  is  the  term 
we  will  use.  Diagnosis  is  a  trivial  process  in  some  in¬ 
stances  especially  in  automatic  process  control.  In  a 
household  temperature  control  system,  the  furnace  is 
always  assumed  to  be  the  culprit  (within  the  purview 
of  the  automatic  controller)  when  actual  temperature 
departs  from  that  which  is  desired.  On  the  other  hand, 
in  a  process  of  any  complexity,  particularly  when  prob¬ 
abilistic  elements  such  as  human  beings  play  a  part, 
diagnosis  can  be  an  extremely  complex  process.  A  time 
overrun  on  an  activity  in  a  PERT  network  seldom  has 
a  simple  cause,  for  example.  In  manufacturing  cost  and 
schedule  control  systems,  allocation  of  blame  between 
performer  and  standards  estimator  has  always  been  a 
source  of  argument,  to  say  the  least.  Diagnosis  may 
occur  before  direction,  in  which  case  causality  is  con¬ 
sidered  in  formulating  the  direction,  or  it  may  occur 
as  a  result  of  the  direction,  in  which  case  it  is  performed 
by  the  controlled  entity. 

Control  systems  differ  as  to  the  specificity  of  the  de¬ 
sired  behavior.  In  simple  cases,  the  purpose  of  control 
is  strictly  regulative,  keeping  performance  within  rea¬ 
sonable  limits.  But  in  other  cases,  again  especially 
when  people  are  involved,  the  control  system  assumes 
an  educative  role.  There  are  two  aspects  to  this  edu¬ 
cative  role.  In  cases  where  direction  is  readily  deter¬ 
minable  and  diagnosis  can  be  more  effectively  per¬ 
formed  at  the  activity  level,  the  control  signals  motivate 
search  (self-diagnosis  so  to  speak).  In  cases  where 
diagnosis  is  performed  prior  to  direction,  the  control 
signals  encourage  desirable  aspects  of  activity  by  “re¬ 
warding”  them  in  some  way  (or  by  “punishing”  unde¬ 
sirable  aspects),  thus  leading  in  theory  to  process  im- 
pro\emcnt,  “learning”  again.  All  incentive  systems, 
whether  applied  to  top  executive  or  to  workers  are 
educative  in  purpose. 

Control ,  just  as  planning ,  is  always  based  on  a 
model.  The  model  may  be  a  simple  expression  of  for¬ 
mal  cause  and  effect  relationships  (e.g.,  furnace  yields 
heat,  bad  supervision  yields  unfavorable  labor  vari¬ 
ance),  or  it  may  be  highly  informal,  implicit  in  post 
mortem  analyses  effected  upon  major  deviations,  or  it 
may  be  an  explicit  mathematical  model. 

Hierarchy  in  planning  and  control 

All  management  organizations  arc  hierarchical  to  a 
degree,*  and  this  implies  hierarchy  in  the  processes  of 
planning  and  control.  Even  at  a  single  organizational 
level  planning  is  “senior”  to  control  in  the  sense  that 
it  provides  the  goal  for  the  control  process.  Another 
potential  interrelation  between  planning  (particularly 
operational  planning)  and  control  occurs  in  the  diag¬ 
nostic  process.  As  the  control  model  is  revised,  pre¬ 


sumably  the  planning  model  should  reflect  the  new 
insights.  We  advocate  that  this  process  be  formalized. 

There  are  several  levels  of  planning  and  control  in 
a  management  organization.  Wc  expect  to  see  at  the 
lowest  level  of  the  system  detailed  plans  (often  in  the 
form  of  schedules)  driving  the  control  of  the  basic 
productive  physical  processes.  At  higher  levels,  we 
expect  to  see  efforts  to  coordinate  (via  a  plan)  the 
control  of  multiple  interdependent  activities.  Lower 
level  managers  control  operating  processes,  but  higher 
level  managers  control  lower  level  managers.  Anthony 
(1965)  draws  a  sharp  distinction  between  “opera¬ 
tional  control”  and  “management  control”  (in  the  lat¬ 
ter  he  is  referring  to  the  manager  as  the  controlled 
entity  as  well  as  the  controller).  We  will  be  par¬ 
ticularly  concerned  with  a  related  aspect,  that  of  con¬ 
trolling  the  planning  process.  Consequently,  wc  draw 
a  sharp  distinction  between  planning  process  control 
and  operating  process  control .** 

On  the  virtues  of  intelligent  management 

Campaigning  for  intelligent  management  does  not 
seem  controversial  on  the  face  of  it,  but  it  does  beg  for 
some  consideration  of  the  economic  justification  of 
our  particular  forms  of  intelligence.  We  seek  better 
understanding — more  valid  models — and  we  seek  bet¬ 
ter  logical  understanding.  A  more  valid  model  means 
more  predictable  execution  of  plans  as  previously 
noted.  Better  logic  in  developing  plans  means  better 
alternative  courses  of  action  selected,  other  things 
being  equal.  In  regulative  control,  a  better  model  en¬ 
ables  reduced  variation  in  performance — output  to 
closer  tolerances,  for  example;  in  educative  control, 
a  better  model  enables  more  accurate  recognition  of 
desirable  behavior  and  hence  more  rapid  improve¬ 
ment. 

The  nature  of  this  recognition  can  be  illustrated  by 
the  story  of  B.  F.  Skinner’s  “superstitious  pigeons” 
(Holland  and  Skinner,  1961,  p.  88).  Several  pigeons 
were  placed  in  separate  boxes.  A  feeding  mechanism 
delivered  food  to  each  pigeon  every  1 5  seconds  re¬ 
gardless  of  what  the  pigeon  was  doing.  After  operating 
in  this  way  for  some  time,  the  experimenter  observed 
that  one  bird  was  sitting  very  still,  another  bowing, 
another  turning  around  in  tight  circles,  another  hop¬ 
ping  on  one  foot,  and  so  on.  Each  bird  repeated  its  own 
ritual  between  feedings.  Analogous  (presumably  civil¬ 
ized)  human  reaction  to  indiscriminant  rewards  or 


*We  are  well  aware  of  even  higher  levels  such  as  planning 
process  control  planning  and  planning  process  control  control, 
and  so  forth.  We  assert  that  an  information  system  sufficient 
for  planning  process  control  is  sufficient  for  higher  levels  as 
well. 


**For  a  thorough  discussion,  see  Zannetos  (1965b). 
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punishment  are  frequently  found  in  competitive  athlet¬ 
ics  and,  we  suspect,  in  management. 

We  also  will  offer  subsidiary  arguments  for  formal¬ 
ity  in  planning  and  control  processes.  One  reason  is 
that  while  various  cognitive  agents,  namely  people,  may 
come  and  go,  in  order  for  the  organization  to  increase 
its  intelligence  over  time,  it  must  make  some  provision 
for  recording  the  accumulated  planning  knowledge.  It 
thus  guards  against  loss  of  memory  together  with  the 
people.  A  formal  model  can  be  stored  in  “memory” 
and  hence  provide  continuity  in  intellectual  growth. 

Specifications  for  intelligent  information  systems 
Introduction 

We  are  focusing  upon  operating  process  control  and 
planning  process  control  as  the  central  cognitive  proc¬ 
esses  of  a  management  organization.  In  operating  proc¬ 
ess  control,  we  wish  mainly  to  obtain  specified  be¬ 
havior  of  the  activities  which  comprise  operations.  In 
planning  process  control,  wc  assume  primarily  an  edu¬ 
cative  goal,  that  is  to  say,  we  wish  explicitly  to  improve 
the  planning  process.  We  will  first  treat  the  problem 
of  regulation  and  improvement  in  operations  and  iden¬ 
tify  therein  the  potential  for  structures  of  different 
“levels  of  intelligence,”  and  we  will  attempt  to  specify 
the  information  system  requirements  implied  for  the 
highest  defined  level  of  intelligence.  Wc  will  then  turn 
to  the  more  difficult  problem  of  planning  process 
control.  The  difficulty  stems  from  less  tangible  process 
goals,  less  formal  processes,  and  unclear  boundaries 
on  the  problems  being  attacked.  The  characteristics  of 
the  control  process  arc  the  same  in  planning  process 
control  as  in  operating  process  control,  but  the  process 
being  controlled  is  sufficiently  different  in  the  former 
that  only  a  highly  intelligent  control  system  will  suffice 
to  assure  improvement. 

Operating  process  control 

There  is  always  a  process  model  which  underlies 
control.  It  is  on  the  basis  of  this  model  that  the  mag¬ 
nitude  and  sign  (in  general,  the  nature)  of  the  control 
direction  is  determined,  and,  in  more  complicated  sys¬ 
tems,  the  particular  agency  to  receive  the  signal  is 
determined. 

The  model  can  be  naive  or  sophisticated.  This  is 
partially  a  question  of  the  complexity  entailed  in  the 
model,  but  in  our  view,  more  fundamentally  related  to 
the  depth  to  which  underlying  cause  and  effect  rela¬ 
tionships  are  captured.  Applying  a  polar  classification 
to  a  continuum,  we  identify  the  extremes  as  symp¬ 
tomatic  and  causal  control.  That  there  is  a  continuum 
of  causality  should  be  clear  to  anyone  who  has  at¬ 
tempted  to  respond  in  good  faith  to  a  three-year-old’s 
infinite  series  of  “whys.”  An  example  of  clearly  symp¬ 


tomatic  control  is  a  wage  incentive  system  used  for 
educative  productivity  control.  Output  and  reward  are 
directly  linked,  and  little  formal  attention  is  paid  to 
causes  of  output  (or  lack  of  output)  except  when 
major  dislocations  such  as  machine  breakdowns  or  ma¬ 
terial  shortages  disrupt  the  process.  The  implicit  as¬ 
sumption  is  made  that  high  output  results  solely  from 
energetic  or  skilled  attention  to  duty  by  the  worker. 
If  this  assumption  is  largely  correct,  the  system  works. 
But  if  output  is  affected  by  a  substantial  number  of 
causes  other  than  the  worker’s  activity,  the  system  can 
be  acrimonious  in  its  administration  and  ineffective 
in  its  application.*  A  more  causally  oriented  control 
system  applied  to  the  same  problem  would  attempt  to 
correct  output  variations  for  “degree  of  difficulty”  so 
to  speak,  by  removing  the  effects  of  differences  among 
tasks  (i.e.,  more  precise  standards),  differences  among 
materials  or  material  suppliers,  differences  among  ma¬ 
chines  and  the  like.  It  would,  in  this  case,  isolate  as 
nearly  as  possible  that  portion  of  output  variation  truly 
attributable  to  the  worker.  In  the  extreme  case,  it 
would  attempt  to  classify  detailed  elements  of  the 
worker’s  behavior  as  being  causally  related  to  output 
and  to  reward  each  appropriately,  which  is  to  say,  it 
would  assist  in  discrimination. 

The  close  connection  between  causality  and  under¬ 
standing  implies  that  causal  control  provides  a  higher 
level  of  intelligence  than  symptomatic  control. 

Another  dimension  of  classification  related  to  the 
question  of  intelligence  is  the  adaptivity  of  the  model. 
At  the  lowest  level  in  this  case  is  reflexive  control , 
which  is  based  on  a  fixed  model  with  fixed  or  externally 
supplied  parameter  values.  The  name  derives  from  the 
parallel  to  a  reflex  in  human  physical  behavior;  the  ef¬ 
fect  is  that  of  a  “stored  response”  to  stimuli.  Habitual 
behavior  is  reflexive.  Reflexive  control  is  employed  in 
very  simple  situations  such  as  the  house  temperature 
control  as  well  as  in  highly  complicated  systems  for 
inventory  control  based  on  mathematical  models.  The 
salient  point  is  that  the  system  behavior  is  not  easily 
modified  and  is  certainly  not  self-adapting  to  a  chang¬ 
ing  environment. 

The  next  level  is  parametrically  adaptive  control . 
At  this  level  the  model  is  fixed  as  to  the  constituency 
of  variables  and  parameters  and  the  relationships  among 
them,  but  the  values  of  the  parameters  are  changed  as 
a  function  of  experience  or  as  exogenously  supplied 
data  vary.t  This  type  of  system  is  often  seen  in  chcmi- 

*The  bailie  among  workers  for  “make  out,”  i.e.,  tasks  for 
which  causes  other  than  the  workers’  effort  make  good  per¬ 
formance  easy,  is  behavior  symptomatic  of  this  type  of  prob¬ 
lem. 

tSuch  systems  can  be  thoroughly  sophisticated.  See  Bellman 
(1961). 
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cal  process  control,  in  which  the  model  relates  yield 
to  a  variety  of  input  and  process  factors.  The  weights 
placed  upon  the  factors  are  varied  as  experience  ac¬ 
cumulates,  providing  lagged  response  to  new  environ¬ 
mental  information.  Another  aspect  of  parametric  adap¬ 
tivity  is  seen  in  inventory  control  based  on  “adaptive 
smoothing.”*  In  this  ease,  the  smoothing  parameters 
are  adjusted  as  data  are  processed  in  an  attempt  con¬ 
tinuously  to  obtain  a  minimum  variance  forecast. 

At  a  still  higher  level,  we  can  hypothesize  inductive 
control ,  in  which  the  entire  model  is  subject  to  change, 
both  in  structure  and  constituency  (i.e.,  functional 
form)  as  well  as  parameter  values.  What  we  have  in 
mind  here  is  continuous  re-evaluation  of  the  model, 
diagnosis  being  the  central  process.  Inductive  control, 
in  attempting  to  establish  causality  in  greater  depth, 
involves  formulation  of  new  hypotheses  and  tests  there¬ 
of.  Indeed,  since  it  is  advancing  “hypotheses  of  causes” 
it  parallels  closely  the  Bayesian  “prior  to  posterior” 
process  with  highly  complex  multivariable  models. § 

We  would  ask  an  additional  step  in  inductive  con¬ 
trol.  Since  the  control  model  is  being  adapted,  it  would 
seem  essential  to  adapt  the  related  planning  model  as 
well.  In  fact,  we  will  denote  inductive  systems  which 
provide  for  direct  updating  of  planning  models  as 
prognostic  as  well  as  diagnostic  in  purpose. 

We  know  of  no  examples  of  formal  inductive  con¬ 
trol  systems  in  operation.  To  clarify  the  ideas,  how¬ 
ever,  consider  the  following  situation.  There  was  a 
statistical  analysis  of  yield  performed  on  a  mechanical 
process.  The  purpose  was  to  relate  process  yield  to 
various  (controllable)  input  and  process  factors  and 
ultimately  to  increase  the  yield.  A  rather  elegant  pre¬ 
dictive  model  was  obtained,  but  the  remaining  unex¬ 
plained  process  variation  was  still  substantial.  Addi¬ 
tional  variables  were  then  studied  to  little  effect  until 
the  time  of  day  during  which  the  process  was  being 
operated  was  tested.  This  variable  showed  almost  domi¬ 
nating  significance.  Further  investigation  showed  that 
the  third  shift  superintendent  was  paying  essentially 
no  attention  to  process  yields  with  predictable  effects. 
While  this  situation  may  illustrate  cither  missing  the 
forest  due  to  overzealous  tree  examination  or  seren¬ 
dipity  (depending  on  what  the  responsibilities  of  the 
analysts  are  assumed  to  have  been),  it  is  also  a  clear 
example  of  economically  significant  increased  under¬ 
standing  resulting  from  an  inductive  control  process. 

t Brown  (1963)  has  a  complete  description  and  discussion. 

§The  “hypothesis  of  causes'’  was  Bayes’  own  name  for  his 
theorem  Basic  references  in  Bayesian  statistical-decision  theory 
are  Schlaifer  (1959)  for  the  layman  and  Raiffa  and  Schlaifer, 
(1961)  for  the  expert.  The  process  of  modifying  functional 
form  and  variable  membership  in  models  is  not  at  present 
included  among  the  problems  solved  by  the  Bayesians,  but 
“adapting”  parameter  values  is. 


We  shall  later  cite  additional  eases  where  the  purpose 
of  inductive  control  is  implied  in  informal  systems. 
Levels  of  intelligence  in  operating  process  control 

As  must  be  clear,  we  would  classify  reflexive-symp¬ 
tomatic  control  at  the  bottom  of  our  scale  and  (prog¬ 
nostic)  inductive-causal  at  the  top,  since  the  latter  pro¬ 
vides  the  potential  for  achievement  of  the  highest  level 
of  intelligence  in  an  organization — for  learning,  in  the 
general  sense. 

But  let  it  be  clearly  understood  that  we  are  not  ad¬ 
vocating  wholesale  redesign  of  all  operating  process 
control  systems  to  achieve  this  sort  of  intelligence.  In 
fact,  to  the  extent  that  the  environment  is  stable  and 
very  well  understood,  a  reflexive  control  structure  may 
be  wholly  adequate.  After  all  there  are  many  eases 
where  response  to  the  symptoms  also  cures  the  disease. 
To  the  extent  that  constituency  and  general  functional 
relationships  are  well  understood  and  stable,  parametri¬ 
cally  adaptive  control  may  be  just  right.  On  the  other 
hand,  to  the  extent  that  the  environment  is  not  per¬ 
fectly  understood  or  is  changing,  then  there  exists  an 
argument  for  inductive  control.*  As  we  view  the  world, 
the  latter  category  appears  to  include  the  majority  of 
system  control  problems  and  the  majority  of  activities 
subject  to  rapid  technological,  political,  or  market 
change. 

We,  therefore,  advocate  intelligent  control  as  a  rule, 
but  admit  to  exceptions  and  add  that  all  types  can 
eo-exist  within  the  same  overall  system.  In  fact,  an 
intelligent  control  system  may  be  viewed  as  having  the 
capability  to  sense  (through  scanning)  what  level  of 
sophistication  should  be  applied  to  each  situation.  As 
we  have  already  mentioned,  planning  and  control  are 
hierarchical  in  nature. 

Information  system  requirements  for 

intelligent  operating  process  control 

An  initial  step  in  establishing  causality  is  to  estab¬ 
lish  association  between  the  basic  variable  or  variables 
of  interest  and  other  factors  capable  of  being  measured 
and  either  corrected  for  or  controlled  themselves.  Thus, 
the  first  step  in  uncovering  the  causes  of  lung  cancer 
has  been  to  establish  the  association  of  the  incidence 
of  that  disease  with  cigarette  smoking.  Association  is 
necessary  for  causality  but  not  sufficient  to  prove  it; 
it  is  a  first  step.T  Since  diagnosis  in  most  eases  occurs 
after  the  fact,  as  with  other  post  mortem  (or  post 

♦There  is  always  a  question,  too,  of  the  economically  justifiable 
depth  of  diagnosis  in  inductive  control.  Since  causality  is  not 
necessary  for  predictability  (it  is  sufficient ),  it  may  be  optimal 
to  cease  searching  at  some  symptomatic  level. 


tWith  rare  exception,  causality  cannot  be  established  statis¬ 
tically;  proof  of  sufficiency  often  requires  systematic  elimina¬ 
tion  of  all  other  possible  causes  or  controlled  experimentation. 
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vie  tori  am )  activities,  it  begs  for  reconstruction,  in  a 
flexible  way,  of  the  situation  when  the  unexpected 
occurred.  This  implies  a  requirement  for  a  variety 
of  associations  among  factors,  temporal  as  well  as  func¬ 
tional,  for  adequate  feeding  of  the  diagnostic  process, 
especially  if  it  is  to  be  carried  out  before  the  unde¬ 
sirable  event  completes  its  course.  Furthermore,  a 
difficulty  in  establishing  association  is  confounding , 
which  is  the  inability  to  separate  the  effects  of  two  or 
more  variables  due  to  overly  gross  or  aggregated  meas¬ 
urement.  Hence,  we  require  in  our  supporting  infor¬ 
mation  system  the  facility  for  functional  and  temporal 
association  with  precision  and  in  detail.  To  be  more 
helpful  this  association  must  take  place  as  soon  as 
possible^  on  the  basis  of  incomplete  information,  and 
be  refined  as  new  data  arc  received. 

The  problem  of  deciding  just  which  measurements 
should  be  maintained  is  difficult.  Potentially  relevant 
data,  not  just  known  relevant  data,  are  needed,  if  the 
model  itself  is  to  be  modified.  This  fact  may  explain 
the  popularity  of  parametrically  adaptive  control  mod¬ 
els;  with  them  at  least  the  data  base  is  well-defined. 

Let  us  attempt  to  be  more  specific  about  the  idea  of 
association.  What  is  needed  is  a  way  of  finding  out  the 
values  of  a  large  number  of  variables  which  were  cur¬ 
rent  at  some  point  in  time.  Functional  association  re¬ 
quires  linkages  among  the  factors  and  the  basic  proc¬ 
ess  measurement.  In  the  yield  analysis  described  above, 
for  instance,  all  of  the  measurements  of  input  and  proc¬ 
ess  characteristics  had  to  be  linked  to  the  yield  on  a 
particular  batch.  The  temporal  association  capability 
allows  for  lagged  effects,  and  for  dynamic  analysis  of 
phenomena  in  general. 

A  detailed  associative  data  base  is  the  raw  material 
for  inductive  control,  but  additional  capabilities  arc 
required  for  the  diagnostic  elements.  Some  aspects  that 
are  known  to  be  present  (but  which  are  not  well  un¬ 
derstood)  are  pattern  recognition  and  pattern  genera¬ 
tion.  The  former  includes  the  ability  to  perceive  rele¬ 
vant  associations  and  to  match  a  given  pattern  to  ob¬ 
served  behavior.  Often  this  involves  “normalizing”  the 
data,  putting  them  in  a  proper  format  or  otherwise 
transforming  them  to  conform  to  the  pattern  or  pat¬ 
terns  being  tested.  For  example,  simply  arraying  data 
in  time  series  form  normalizes  them  for  certain  dynamic 
pattern  matching;  in  other  eases,  a  graphing  of  fre¬ 
quency  spectra  might  be  required. 

Pattern  generation  is  even  less  well  understood,  but 
it  clearly  involves  abstraction  and  quantitative  hypoth¬ 
esis  formulation,  which  is  to  say,  model  building.  The 
question  begged  is  what  is  the  source  of  the  model. 
Much  opinion  suggests  that  there  exist  frameworks,  gen¬ 
eral  theories,  or  taxonomies — broad  categorizations  of 
phenomena — which  suggest  detailed  models  for  testing. 


Freudian  psychology,  Marshallian  economic  theory,  or 
more  recently,  Forrester’s  “industrial  dynamics” 
(1962),  are  examples  of  formal  frameworks.  In  gen¬ 
eral,  however,  the  totality  of  one’s  experience,  observa¬ 
tion,  and  education  serves  as  the  framework  for  a 
human.  Pounds  (1965)  suggests  that  the  process  of 
diagnosis  often  begins  (a  “problem  is  found”)  when 
behavior  departs  from  that  suggested  by  one  of  these 
frameworks. 

An  example  may  serve  to  clarify  what  we  mean  by 
pattern  recognition  and  generation.  Forrester  (1962) 
has  cited  some  instances  of  self-induecd  oscillatory 
behavior  in  business,  one  of  which  was  evidenced  by 
inexplicable  seasonal  demand  for  a  consumer  product. 
By  drawing  on  his  general  framework  he  was  able  to 
construct  a  model  which  (qualitatively)  matched  that 
of  the  (normalized,  measured)  behavior  of  the  firm. 
From  his  model,  he  was  able  to  deduce  that  the  cause 
of  the  seasonal  peaks  and  valleys  were  the  firm’s  tradi¬ 
tional  promotional  patterns  and  its  customers’  anticipa¬ 
tion  of  this  pattern. 

Pattern  recognition,  abstraction  and  hypothesis  for¬ 
mulation  remain  shrouded  in  mystery  as  to  their  pre¬ 
cise  mechanisms;  they  are  apparently  tied  up  with  the 
very  most  arcane  human  capabilities  which  are  often 
collectively  labeled  “creativity.”*  The  mystery  notwith¬ 
standing,  we  require  these  faculties  as  operative  ele¬ 
ments  in  inductive  control.  Also  bear  in  mind  that 
they  must  be  employed  in  the  worst  of  all  possible 
inferential  worlds  evidencing  as  it  docs  probabilistic  and 
dynamically  non-stationary  behavior  and  imperfect 
measuring  devices.  And,  given  the  mystery,  we  would 
expect  humans,  as  opposed  to  machines,  to  supply  the 
capabilities  required. 

Planning  process  control 

When  the  planning  process  is  brought  under  sur¬ 
veillance,  all  of  the  previously  cited  aspects  of  control 
apply,  but  there  are  some  new  problems  to  face  as 
well.  Some  of  these  can  be  stated  as  follows: 

•  Planning  is  always  based  on  a  model,  so  in  control 
of  planning  there  is  a  metamodeling  problem.  We 
require  a  model  of  the  (planner’s)  modeling  proc¬ 
ess  and  a  model  of  the  employment  of  the  planning 
model. 

•  Planning,  especially  strategic  planning,  is  often 
based  on  information  about  matters  external  to 
the  firm  or  organization.  For  example,  predic¬ 
tions  of  competitors’  behavior  or  general  economic 
conditions  are  often  basic  to  commercial  planning. 

♦Minsky  (1963)  and  Newell  and  Simon  (1961)  have  much 
to  say  on  this.  The  fact  is  that,  at  this  point  in  time,  people 
can  do  these  things  very  well  and  machines  not  well  at  all. 
See  also  Licklider  (1965). 


I 
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The  enemy’s  order  of  battle  oeeupies  a  similar 
position  in  military  planning.  Henee,  planning 
proeess  eontrol  requires  a  data  base  that  is  not 
neeessarily  a  convenient  by-produet  of  operating 
proeess  eontrol  or  otherwise  at  the  disposal  of  its 
users. 

•  In  order  to  establish  eontrol,  there  must  be  a 
proeess  goal  or  standard,  in  this  ease  an  objective 
purpose  for  planning.  Yet  it  is  not  always  abun¬ 
dantly  elear  just  what  one  is  attempting  to  aehieve 
by  planning,  with  the  possibly  innoeuous  obser¬ 
vation  that  he  is  attempting  to  bring  about  some 
order  to  an  otherwise  ehaotie  situation  and  so 
structure  his  problem. 

•  Planning  is  frequently  intuitive  and  subjective 
both  as  to  proeess  and  to  data.  Yet  in  order  to 
exert  eontrol,  the  subjective  estimates  and  value 
judgments  require  quantification  and  their  func¬ 
tional  relationship  with  the  planning  goal  requires 
establishment. 

•  Planning,  in  many  eases,  looks  far  into  the  future. 
It  would  be  desirable  to  eonduet  post  mortem 
analysis  for  proeess  improvement,  yet  if  the  plan¬ 
ning  proeess  controller  waits  for  the  future  to  re¬ 
veal  itself  completely,  the  eontrol  eyele  time  will 
be  too  long. 

•  Planning  is  typieally  a  group  rather  than  an  in¬ 
dividual  proeess.  We  understand  little  enough 
about  individual  behavior  but  even  less  about 
group  behavior.  Operating  also  is  often  a  group 
proeess,  but  planning  is  “groupthink”  rather  than 
“groupdo.”  Again,  this  is  a  matter  of  difficulty 
in  associating  eause  and  effeet. 

•  Correlatively,  planning  is  a  task  still  (if  tem¬ 
porarily)  performed  largely  by  people.  Attempts 
to  observe  or  to  experiment  with  people  often 
lead  to  the  well-known  “Hawthorne  Effeet,”  in 
whieh  the  subjects  respond  to  the  fact  or  condi¬ 
tions  of  the  experiment  rather  than  to  the  en¬ 
vironment  being  studied.*  What  is  even  worse, 
often  the  environment  surrounding  the  experi¬ 
ment  itself  ehanges  by  the  mere  faet  of  being 
observed. 

This  list  is  long  enough  to  be  discouraging  to  the 
most  ardent  of  idealists;  but  the  alternative  of  uncon¬ 
trolled  planning  should  be  sufficiently  dismaying  to 

♦The  name  stems  from  some  working  condition  experiments 
conducted  at  the  Hawthorne  Works  of  Western  Electric  in 
the  thirties.  A  group  of  women  workers  was  submitted  to 
varying  lighting,  healing  and  other  factors.  Regardless  of 
conditions  their  productivity  rose.  The  experimenters  finally 
concluded  that  the  women  were  responding  to  the  attention 
that  accompanied  the  experimentation.  Described  in  Roeth- 
lisberger  and  Dickson  (1939). 

make  the  effort  worthwhile.  And  this  list  suggests  a 


proeess  sufficiently  poorly  understood  to  require  in¬ 
dued  ve-eausal  eontrol. 

Information  system  requirements 

for  planning  process  control 

We  elearly  require  an  upgraded  data  base  for  plan¬ 
ning  proeess  eontrol.  Its  seope  requires  expansion  to 
inelude  both  external  data  (ineluding  foreeasts  of  ex¬ 
terna!  variables)  and  subjective  data.  By  the  latter, 
we  mean  that  the  planning  assumptions,  subjective 
estimates,  and  value  judgments  should  be  formally 
recorded.  And,  of  eourse,  we  require  the  same  asso¬ 
ciative  facility  with  these  data  as  we  did  for  operating 
proeess  eontrol. 

What  this  amounts  to  is  a  plea  for  formal  models 
in  planning  whieh  we  add  to  those  previously  voieed 
by  others,  notably  Emery  (1965).  The  added  motiva¬ 
tion  is  the  potential  here  for  planning  process  improve¬ 
ment.  To  this  we  add  the  requirement  of  formal  goals 
for  the  planning  proeess. 

For  eontrol,  it  is  not  sufficient  merely  to  evaluate 
the  product — the  plan — we  require  aeeess  to  the  proe¬ 
ess  as  it  operates.  This  means  somehow  eapturing  the 
“stream  of  consciousness”  of  the  planner  to  obtain 
his  “traee,”  i.e.,  the  logie  he  has  used  to  formulate  his 
plan.  To  tighten  the  loop  in  long  range  planning  we 
need  a  method  for  analyzing  incomplete  returns,  to 
infer  on  the  basis  of  partial  data.  And,  we  require  as 
before  a  powerful  diagnostic  facility.  Finally,  some 
provision,  such  as  clandestine,  unexpected,  or  con¬ 
stant  surveillance  must  be  made  to  avoid  the  Haw¬ 
thorne  Effeet. 

Commentary  on  the  current  state  of  the  art 

Introduction 

We  will  attempt  to  review  what  we  pereeive  to  be 
examples  of  elements  of  intelligent  information  sys¬ 
tems  whieh  are  generally  or  specifically  in  operation 
We  face  a  problem  in  so  doing  in  that  we  suspect  that 
organizations  whieh  have  achieved  higher  levels  of 
intelligence  in  their  information  systems  are  probably 
intelligent  enough  not  to  publicize  the  faet,  so  the 
state  may  not  be  so  primitive  as  we  represent  it. 

The  state  of  operating  process  control  systems 

Control  systems  for  detailed  productive  processes 
have  been  growing  in  sophistication  ever  since  com¬ 
puters  beeame  generally  available.  In  continuous 
proeess  eontrol,  for  example,  very  elaborate  formal- 
model  based  systems  for  ehemieal  processing  are  now 
eommon.  These  vary  in  their  complexity,  but  most 
commercially  available  systems  are  eapable  of  multi- 
variable  eontrol  at  multiple  levels  (i.e,  they  adjust  the 
proeess  to  eonform  with  desired  values  on  several 
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variables  and  also  compute  the  desired  values  for  the 
variables  based  on  external  inputs).  Parametrically 
adaptive  systems  of  at  least  modest  scope  are  opera¬ 
tive  as  well.  However,  since  these  systems  are  fixed 
as  to  model  structure,  such  inductive  inference  towards 
model  improvement  as  takes  place  must  be  performed 
externally  to  the  system.  These  systems  are  of  interest 
as  models  for  man-maehine  system  control,  but  while 
sophisticated  in  model  structure,  they  offer  no  guide 
for  model  improvement — they  do  not  evolve. 

Another  area  of  interesting  development  is  that  of 
detailed  job  shop  production  scheduling  and  control 
as  practiced  in  Hughes  Aircraft  (Steinhoff,  1966)  and 
Westinghouse  Electric  (Trilling,  1964),  among  other 
firms.  The  general  structure  of  these  systems  is  this. 
A  simulation  model  is  fed  with  inputs  of  the  current 
order  backlog  (with  routings,  processing  time  esti¬ 
mates,  and  due  dates),  the  shop  configuration  (ma¬ 
chines,  and  men),  and  some  decision  rules  for  dis¬ 
patching  the  jobs.  The  model  is  run  and  rerun,  simulat¬ 
ing  the  future  course  of  events,  allowing  for  adjust¬ 
ments  to  backlog  (i.e.,  subcontracting),  or  capacity 
(overtime,  added  shifts)  and  the  decision  rules.  When 
a  “satisfactory”  simulation  is  obtained,  the  simulated 
start  time  of  each  job  on  each  machine  is  used  as  the 
scheduled  start  time  for  the  job  in  the  shop.  This 
schedule  provides  the  plan  for  production  control. 

These  ‘‘finite  capacity”  schedulers  (so-ealled  be¬ 
cause  they  explicitly  consider  the  capacity  available 
at  each  work  station  before  simulating  the  assignment 
of  a  task)  are  considerably  more  complex  than  the 
standard  “infinite  capacity”  scheduling  systems  in 
which  a  scheduled  date  for  each  task  is  obtained  by 
dating  back  from  the  job  due  date  using  “standard 
lead  times”  (which  allow'  for  direct  production  time, 
waiting,  transit,  setup  and  the  like)  for  each  operation 
on  the  routing.* 

One  problem  with  infinite  capacity  schedules  is 
that  they  are  generally  not  feasible,  even  in  theory. 
The  schedule  dates  provide  crude  targets  for  progress, 
but  unless  work  station  capacity  is  directly  considered, 
a  deviation  from  schedule  may  only  signify  that  the 
schedule  was  impossible  at  the  outset.  Then,  devia¬ 
tions  resulting  from  model  inadequacy  are  confounded 
with  true  process  deviations.  This  is  less  a  problem 
with  a  finite  scheduler.  A  deviation  in  the  latter  ease 
generally  indicates  that  something  unexpected  has  oc¬ 
curred  such  as  low  productivity,  a  bad  processing  time 
estimate,  a  material  shortage,  or  failure  to  follow  the 
scheduled  sequence.  While  causality  is  not  pinpointed, 
a  point  of  departure  is  established.  (Even  in  finite 
capacity  schedules,  minor  deviations  tend  to  compound 

♦Emery  (1961)  provides  a  discussion  in  depth  of  various 
alternative  scheduling  systems  including  lhese  two. 


after  a  time  and  schedule  infeasibility  again  rears  its 
head.  Potentially,  this  schedule  “decay”  can  be  cured, 
and  discrimination  of  causes  materially  improved  in 
on-line,  real-time  control  systems.  This  possibility  is 
discussed  in  the  next  section.) 

The  more  detailed  model  (derived  from  the  more 
detailed  data  base)  used  in  finite  scheduling,  and  the 
built-in  time-based  association  of  resources  (work 
stations)  with  activities  provides  the  increased  control 
power.  In  effect,  the  better  model  eliminates  “noise” 
from  the  information  system  and  gives  the  manager 
more  confidence  in  the  deviation  signal.  In  comparison, 
the  naive  infinite  capacity  schedule  tends  to  “cry 
wolf,”  leading  to  ineffective  remedial  action. 

In  conventional  accounting  control  systems  the  state 
of  the  art  is  rather  primitive.  Budgets  and  other  stand¬ 
ards  are  frequently  almost  arbitrarily  arrived  at  and 
often  used  on  a  memorandum  basis.  Few  deviations 
have  any  significance  and  they  could  easily  result  from 
factors  totally  beyond  the  aegis  of  the  controlled  en¬ 
tity.  There  is  no  systematic  way  of  filtering  noise  from 
the  information  system  and  no  aids  are  provided  for 
causal  diagnosis  (beyond  a  superficial  level  or  even  for 
determining  significance).  This  is  not  to  say  that 
managers  do  not  attempt  to  determine  causes  of  budget 
overruns,  for  example  but,  that  such  diagnosis  occurs 
separately  from  the  control  system  (and  in  some  cases, 
in  spite  of  the  control  system).  Often,  because  of 
traditional  accounting  practices,  data  are  averaged  in¬ 
discriminately  and  also  “elosed-out”  destroying  the  use¬ 
fulness  of  the  data  base  for  effective  managerial  con¬ 
trol  and  decision  making.  We  must  stress  that  we  are 
not  concerned  here  with  the  external  financial  reporting 
aspects  of  accounting  in  which  justification  for  the 
conventional  practices  may  be  found.  Our  focus  is 
internal  control.  For  this,  at  best,  in  the  absence  of 
managerial  brilliance,  the  conventional  wisdom  in  ac¬ 
counting  control  amounts  to  symptomatic-reflexive 
control,  the  bottom  rung  of  our  intelligence  ladder.  It 
is  small  wonder  that  managerial  behavior  approximating 
that  of  the  “superstitious  pigeon”  is  not  uncommon. 

There  are,  however,  some  candles  being  lit  in  this 
area  of  stygian  darkness.  The  general  idea  of  the 
“flexible  budget,”  where  elementary  cause-effect  rela¬ 
tionships  are  derived  to  allow  measurement  and  separa¬ 
tion  of  deviations  from  plan  due  to  volume  of  activity 
(“volume  variance”)  from  deviations  due  to  efficien¬ 
cy  in  labor  performance  and  use  of  material  resources 
independent  of  volume,  represents  an  attempt  to 
separate  gross  uncontrollable  (by  the  production 
manager  in  this  case)  effects  from  those  which  can 
properly  be  laid  at  his  door.  But  the  accountants  out¬ 
side  of  limited  use  of  this  tool  (in  the  area  of  manu¬ 
facturing  operations)  have  not  pressed  on  in  this 
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direction,  neither  in  depth  nor  in  order  to  apply  these 
techniques  to  other  aspects  of  operations. 

More  encouraging  are  the  recent  attempts  in  the 
Bell  System  (Harvey,  1966),  the  military  and  else¬ 
where  (Black,  n.d.;  Zschau,  1964)  to  establish  per¬ 
formance  standards  on  the  basis  of  more  precise 
statistical  models.  For  example,  suppose  there  is  a 
multiple  plant  company  with  each  plant  producing 
comparable  products.  An  electrical  utility  will  serve 
as  an  example.  No  doubt  there  will  be  variation  among 
plants  in  measures  of  performance,  say  average  deliv¬ 
ered  cost  per  kilowatt  hour.  Some  of  this  variation 
is  attributable  to  plant  management,  but  a  number 
to  factors  arc  outside  the  control  of  management,  such 
as  fuel  costs,  age  of  generating  equipment,  population 
density,  climate,  size  of  the  area  served,  industrial  con¬ 
centration  and  classification  and  so  on.  Merely  to 
compare  plants  on  the  basis  of  the  raw  performance 
measure  is  patently  unfair  to  the  plant  manager  who 
has  drawn  unfavorable  circumstances.  Mean  perform¬ 
ance  as  a  function  of  all  of  the  uncontrollable  factors 
can  be  predicted  for  a  particular  plant  on  the  basis  of 
statistical  (i.e.,  multiple  regression*)  models.  This 
revised  performance  figure  represents  a  standard  cali¬ 
brated  for  “degree  of  difficulty,1’  so  to  speak.  Perform¬ 
ance  deviations  from  this  calibrated  figure  represent 
true  “managed”  performance  plus  a  much  smaller 
“unexplained”  component,  and  certainly  provide  a 
more  accurate  basis  for  learning,  reward  or  castigation. 

These  control  systems,  in  their  separation  of  un¬ 
controllable  causes  from  controllable,  represent  a  ma¬ 
jor  step  towards  causally  oriented  operating  process 
control.  While  the  particular  techniques  used  in  the 
cited  studies  may  have  been  limited  so  far  mostly  to 
relatively  homogenous  product  oriented  industries,  the 
philosophy  underlying  the  use  appears  impeccable  to 
us,  and  can  be  effectively  extended,  we  believe,  to 
many  other  situations.  For  example,  the  causes  of  vari¬ 
ation  in  overhead  costs  and  central  services  in  general 
may  be  estimated  in  this  manner.  Another  interesting 
possibility  for  extension  would  be  to  associate  the 
residual  variance  with  identifiable  management-con¬ 
trolled  as  well  as  extra  organizational  variables,  e.g., 
work  force  composition,  salary  levels,  some  quantified 
aspects  of  operating  strategy,  competitive  reaction,  etc. 
Associations  of  this  sort  would  lead  naturally  to  diag¬ 
nosis — inductive  control. 

We  have  observed  efforts  toward  diagnosis  in  sys¬ 
tem  management,  particularly  in  the  PERT-based 
planning  and  control  system  employed  by  NASA  in 
the  APOLLO  program.  First,  there  is  a  formal  model 
used  for  planning  and  control.  Second,  the  evidently 

♦Broad  coverage  of  the  technique  can  be  found  in  Ezekial  and 
Fox  (1959)  and  Graybill  (1961). 


widespread  doctrine  of  “visibility”  is  employed  in 
project  time  and  cost  (and  to  a  lesser  degree,  technical 
performance)  control.  This  calls  for  focusing  atten¬ 
tion  on  responsible  parties  in  cases  of  unfavorable  de¬ 
viation  from  plan.  From  discussions  with  both  the  sys¬ 
tem  managers  and  contractors  it  seems  clear  that  what 
goes  on  in  the  “control  rooms”  during  the  post  mor¬ 
tem  project  reviews  is  causally  oriented  diagnosis  to  a 
substantial  degree.*  The  significant  point  is  that  the 
whole  information  system  appears  to  be  oriented  to¬ 
wards  this  process.  We  feel  that  “visibility”  insofar  as 
it  encourages  diagnosis,  is  a  useful  system  design  con¬ 
cept. 

In  addition,  several  attempts  have  been  made  by 
people  at  NASA  to  “calibrate”  the  bias  of  the  plan¬ 
ning  estimates  of  the  contractors  and  thence  to  cor¬ 
rect  the  plan  for  this  bias  as  it  becomes  known,  a 
clearly  prognostic  exercise. 

The  state  of  planning  process  control 

In  general,  it  appears  to  us  that  organizations  have 
recognized  the  need  for  planning  process  control  for 
years,  but  have  instituted  little  or  no  formal  surveil¬ 
lance.  For  example,  there  is  usually  some  control  ex¬ 
ercised  over  the  process  of  budget  preparation  in  gov¬ 
ernment  and  industry,  in  the  form  of  critiques  of  as¬ 
sumptions  and  also  end-of-fiseal-year  post  mortems. 
One  of  the  clearest  examples  of  this  is  the  Westing- 
house  Electric  “Profit  Planning”  system  described  by 
Evans  (1959).  High  level  (product  department)  plans 
arc  examined,  reviewed,  and  critiqued  on  the  basis  of 
their  assumptions  and  substance  before  the  execution 
begins,  and  periodically  during  the  year,  the  execution 
is  reviewed.  Care  is  taken  to  separate  effects  due  to 
poor  planning  or  poor  forcasting  from  those  due  to 
poor  performance  both  by  dialogue  and  in  the  struc¬ 
ture  of  the  planning  accounts  themselves.  By  classify¬ 
ing  costs  into  “committed”  (i.c.^  fixed,  not  under 
management  control),  “managed”  (i.e.,  discretionary 
overhead  such  as  management  salaries,  consultants, 
computer  rental)  and  “product”  (i.e.,  direct  and  in¬ 
direct  materials  and  labor),  greater  precision  in  attribut¬ 
ing  variance  to  particular  gross  causes  is  obtained.  But 
there  is  no  evidence  of  formal  diagnosis  being  employed 
in  this  approach,  and  the  control  system  is  of  an 
ad  hoc  nature.  It  represents,  in  a  sense,  symptomatic 
planning  process  control. 

Another  area  in  which  planning  process  control  has 
evidently  been  pursued  has  been  in  the  military.  The 
classic  doctrine  of  von  Clausewitz,  which  continues  to 

tThe  first  item  on  the  agenda  is  naturally  an  inquiry  into 
what  can  be  done  to  correct  the  deviation  as  it  exists,  only 
then  comes  the  “Why?”  In  many  situations  diagnosis  cannot 
take  place  until  after  some  action  is  taken. 
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influence  military  planning  all  over  the  world,  requires 
the  commander  to  state  formally  his  goals  and  concepts 
and  “estimate  of  the  situation’’  at  the  outset  of  the  plan¬ 
ning  process.  Such  a  system  forces  an  explicit  state¬ 
ment  of  premises  and  conclusions  and  facilitates  after- 
the-fact  assessment  of  blame  among  assumptions,  plan, 
and  execution.  Confounding  the  causes  of  poor  perfor¬ 
mance  is  avoided.  More  recently  these  traditional  meth¬ 
ods  of  military  planning  have  been  extended  and  also 
enriched  by  the  use  of  simulation  and  game-theoretic 
approaches.  While  planning  process  control  has  been 
the  objective  of  many  of  these  improvements,  it  has 
not,  as  far  as  we  can  assess,  become  a  part  of  a  reg¬ 
ular  feedback  control  system.  No  doubt  the  necessity 
for  determining  strategy  on  a  decentralized  basis  (in 
the  field)  has  some  bearing  on  this  issue.  Also  the 
objectives  of  warfare  are  much  simpler  relatively  speak¬ 
ing  than  those  of  the  firm  (we  arc  talking  in  terms  of 
ease  of  definition  not  in  terms  of  execution,  magnitude, 
or  significance),  so  finding  a  substitute  for  an  ad  hoc 
planning  diagnosis  may  not  be  as  critical  for  the  mili¬ 
tary. 

Planning  process  control  has  often  been  employed  in 
military  training;  it  is  not  so  clear  that  it  occurs  under 
the  pressure  of  actual  operations.  But  the  critiques  of 
maneuvers  and  large  scale  training  exercises  frequently 
focus  on  the  planning  process  itself  as  distinct  from 
operations  as  executed. 

Informal,  qualitative  planning  process  control  is  lim¬ 
ited  in  its  effectiveness,  again  because  of  the  discrimina¬ 
tion  problem.  It  is  one  thing  to  know  that  an  estimate 
was  bad;  it  is  another  to  know  why  it  was  bad.  And, 
because  the  planning  process  itself  is  relatively  un¬ 
structured,  it  is  difficult  to  pinpoint  the  particular  sub¬ 
processes  that  were  defective. 

As  we  argued  earlier,  one  way  to  improve  the  po¬ 
tential  of  the  control  process  is  to  move  towards  more 
formal  planning  models.  Rigid  plan  formats  (the  “five 
paragraph”  military  format,  and  the  Westinghouse 
chart  of  planning  accounts  are  examples)  and  specific 
procedures  are  steps  in  this  direction.  We  believe  that 
more  detailed  and  more  complex  models — in  short, 
mathematical  or  computer  models  will  be  of  even 
greater  value  in  forcing  explicit  assumptions  and  esti¬ 
mates  and  organizing  the  process  for  controllability. 

One  example  of  a  trend  in  this  direction  is  the 
Apollo  Project  simulation  model  recently  installed  for 
the  NASA  Office  of  Manned  Space  Flight.*  Because 
the  planning  assumptions  are,  in  effect,  inputs  to  a 
computer  program,  they  are  “visible”;  because  the 
planning  procedure  involves  explicit  recourse  to  a  com¬ 
puter  program  to  examine  alternative  procedures,  it 
would  be  possible  to  obtain  a  “trace”  of  the  search 
process.  Hence,  the  raw  material,  i.c.,  the  basic  “meas¬ 


urements”  of  the  planning  process  arc  available.  But 
since  the  planning  horizon  for  APOLLO  is  long  (at 
least  to  1970),  the  problem  is  obtaining  feedback  on 
planning  results,  and  one  of  accurate  association.  This 
is  a  case  similar  to  that  facing  many  non-military  or¬ 
ganizations,  and  one  which  calls  for  partial  data 
analysis. 

Another  element  of  our  specifications  is  being  im¬ 
plemented  at  Westinghouse  Electric,  namely  that  call¬ 
ing  for  expanding  the  corporate  data  base  to  include 
extra-corporate  data.t  This  also  contributes  basic  meas¬ 
urements  for  planning  process  control. 

Some  conclusions  on  the  state  of  the  art 

Wc  conclude  from  our  brief  review  that  there  exists 
no  publicized  comprehensive  realization  of  intelligent 
management  information  systems.  We  also  perceive  evi¬ 
dence  that  intelligence  is  sought  in  numerous  cases. 

Diagnostic  control  of  operating  processes  seems 
imminent  in  restricted  environments  such  as  continuous 
process  control  and  the  basic  pattern  is  being  estab¬ 
lished  even  in  such  messy  discrete  process  control 
areas  as  job  shop  production  control.  Employment  of 
well-established  statistical  methodology  holds  promise 
for  inductive  control  at  higher  operating  levels. 

Planning  process  control  at  present  is  in  some  cases 
performed,  but  always  performed  informally.  Diag¬ 
nosis  appears  to  be  ad  hoc  and  somewhat  political  in 
flavor.  It  is  at  best  qualitatively  based  and  this,  coupled 
with  the  generally  informal  nature  of  the  control  proc¬ 
ess,  lead  us  to  suspect  that  its  effects  arc  impermanent 
even  when  it  is  effective.  But  the  general  trend  towards 
more  formal  planning  models  offer  opportunity  for 
greater  sophistication  in  control. 

In  total,  many  of  the  bits  and  pieces  from  which 
higher  level  intelligence  in  management  information 
systems  can  be  fabricated  exist.  The  problem  is  to 
assemble  these  within  one  organization. 

Steps  towards  realization  of  intelligent 
information  systems 

Introduction 

The  total  realization  of  intelligent  information  systems 
requires  considerable  research  and  development  before 
it  can  be  accomplished.  For  example,  there  exists  al¬ 
most  no  general  theory  of  diagnosis  in  particular,  and 
inductive  inference  in  general.  On  the  other  hand,  we 
feel  that  some  major  improvements  in  the  state  of  the 
art  could  be  effected  simply  by  recognizing  the  value 

*The  deiails  of  this  system  have  not  yet  been  publicized.  It 
was  designed  by  Peat,  Marwick,  Livingston  and  Company  under 
contract  number  NASw-1223  and  installed  this  year 

tDescribed  in  Burck  (1965,  p.  113). 
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of  what  we  have  called  intelligence  and  reorienting  the 
information  system  to  the  end  of  acquiring  it.  Also,  new 
information  technology  (including  modeling  approaches 
subsumed  under  operations  research  such  as  digital 
simulation  and  heuristic  programming,  as  well  as 
“third  generation”  computer  technology  enabling  real¬ 
time  data  processing  and  time-sharing)  now  affords 
some  major  capabilities  which  can  be  exploited  for 
this  purpose. 

Therefore  (with  perhaps  more  alliteration  than  ac¬ 
curacy)  we  have  defined  our  steps  toward  realization 
as  recognition  and  reorientation,  real-time  processing, 
and  research. 

Recognition  and  reorientation 

Given  the  desire  to  increase  the  intelligence  of  an 
organization,  there  arc  some  fundamental  steps  that 
can  be  taken.  In  our  view,  the  major  discrepancy  be¬ 
tween  typical  operating  process  control  systems  and 
those  which  we  want  is  in  the  explanatory  power  of 
the  underlying  process  models.  That  is  to  say,  conven¬ 
tional  operating  process  control  fails  to  get  at  the  un¬ 
derlying  causes  of  process  variance.  Relatively  unso¬ 
phisticated  statistical  analyses,  such  as  analysis  of 
variance  and  covariance  or  multiple  regression,  can 
shed  considerable  light  in  this  area  in  many  situations. 
And  we  suspect  that  simple  classification-association 
would  uncover  some  of  the  grosser  causes.  For  ex¬ 
ample,  we  recently  observed  a  ease  in  which  a  sales¬ 
man’s  pricing  misbehavior  was  uncovered  simply  by 
comparing  his  customer  claims  experience  with  that 
of  the  rest  of  the  sales  force.  The  basic  statistics 
showed  that  his  claims  were  far  more  frequent  and, 
on  the  average,  larger  than  those  of  his  colleagues. 
Deeper  investigation  uncovered  the  fact  that  many  of 
his  claims  were  unrelated  to  damaged,  missing,  or 
substandard  goods,  but  were  simply  his  mechanism  for 
granting  price  concessions  to  customers  (his  commis¬ 
sions  were  not  adjusted  for  claims).  We  could  cite 
numerous  other  examples  of  surprise  resulting  from 
attempts  to  rationalize  the  causes  underlying  other  per¬ 
formance  measurements.  It  is  important  that  we  stress, 
however,  that  such  rationalization  of  causes  must 
become  part  of  the  information  system  if  intelligent 
system  behavior  is  to  be  obtained. 

A  present-day  managerial  accounting  system  has  the 
capability  to  store  raw  data  and  classify  them.  Classifi¬ 
cation  is  done  purely  on  the  basis  of  human  interven¬ 
tion,  because  the  system  docs  not  have  self-organizing 
characteristics.  But  given  the  classification,  through 
a  matching  process  the  system  can  extract  differences, 
which  are  then  reported  to  management.  But  the 
matching  process  does  not  at  present  tolerate  any 


ambiguity.  It  is  deterministic  in  terms  of  the  classifica¬ 
tions  assigned. 

A  difference  by  itself,  of  course,  does  not  mean  very 
much.  Although  “red”  variances  (debit  balances  in 
manufacturing  accounts,  for  example)  arc  automatically 
considered  undesirable  and  “black”  variances  desirable, 
in  reality,  much  more  analysis  is  necessary  beyond 
this  stage  to  determine  significance.  At  best  these  dif¬ 
ferences  may  point  to  a  potential  problem.  The  ques¬ 
tions  that  come  to  mind  on  observing  these  accounting 
variances  are  mainly  of  two  types:  (a)  how  significant 
(in  a  probabilistic  sense)  arc  they,  and  (b)  what  do 
they  mean? 

To  enlarge  the  capacity  of  the  data  base  and  the 
capabilities  of  the  information  systems,  one  may  store 
in  the  data  base  cues  for  automatic  response  at  the 
operating  level.  This  response  may  be  purely  of  the 
reflexive  control  type  or  the  result  of  elementary 
analysis  performed  by  the  system  itself.  Still,  such  a 
system  docs  not  allow  for  any  ambiguity,  it  is  deter¬ 
ministic  and  inflexible.  To  generate  intelligent  be¬ 
havior,  the  data  base  must  be  capable  of  resolving 
ambiguity,  and  possess  understanding  and  learning 
capabilities. 

We  mentioned  above  in  conjunction  with  the  output 
of  present  accounting  systems  that  the  latter  do  not 
tell  how  important  arc  the  variances  they  generate.  It 
is  all  left  to  the  imagination  of  the  manager.  One  ob¬ 
vious  improvement,  therefore,  is  to  introduce  probabil¬ 
ity  distributions  into  the  data  base.  Also  decision  rules 
for  determining  the  probabilistic  significance  of  the 
observed  deviations  should  be  included.  The  system  can 
be  instructed  to  sift  through  the  differences,  take  reme¬ 
dial  action  on  the  basis  of  prestored  cues,  or  else  report 
the  significant  variations  to  the  manager — “manage¬ 
ment  by  exception.”  But  we  need  not  stop  here.  We 
can  also  in  a  Bayesian  sense  review  the  models  which 
govern  the  expectation  of  system  behavior,  and  also 
update  the  relevant  probabilistic  distributions. 

To  facilitate  understanding  and  improve  learning  it 
is  not  sufficient  to  have  a  system  which  separates  the 
significant  from  the  insignificant  variations  in  perfor¬ 
mance.  Somehow'  the  system  must  help  the  manager 
focus  on  the  underlying  cause-effect  relationships. 

A  method  of  introducing  the  necessary  capability  of 
cause  determination  in  the  data  base  is  to  store  pre¬ 
determined  functional  (cause-effect)  relationships  and 
explicit  decision  rules  to  facilitate  their  use.  Such  an 
arrangement,  however,  is  not  very  different  from  re¬ 
flexive  control;  it  is  inflexible  and  limited  in  its  in¬ 
telligence.  No  understanding  or  inference  takes  place.  * 
We  could  alternatively  “instruct  the  data  base”  in  the 
methods  of  arriving  at  hypotheses  of  cause  and  effect 
relationships  by  itself  (not  a  trivial  task).  This  is  a 


162  Information  System  Science  and  Technology 


more  promising  avenue  because  it  permits  adaptability. 

Simple  and  naive  techniques  such  as  statistical  vari¬ 
ance  and  covariance  analysis,  if  performed  on  the  ac¬ 
counting  variances,  can  yield  useful  cause  and  effect 
relationships  to  be  stored  in  the  data  base  for  further 
analysis  and  testing  hypotheses.  This  type  of  a  system 
was  elsewhere  called  a  functional  accounting  system 
(Zannetos,  1966d)  and  many  of  its  charactcrists  and 
prerequisites  for  implementation  have  already  been 
discussed  (Zannetos,  1966a,  1964,  1965c).  We  believe 
that  with  the  present  state  of  technology  and  knowl¬ 
edge,  the  functional  accounting  system  is  realizable 
now.  Furthermore,  under  such  a  system  many  facets  of 
the  design  of  organization  structures  are  brought  with¬ 
in  the  purview  of  the  system  and  resolved  analytically 
for  the  first  time  (Zannetos,  1965a). 

The  major  discrcpancv  in  planning  process  control 
derives  from  the  informality  both  of  the  planning  proc¬ 
ess  itself  and  whatever  planning  review  procedures 
exist.  The  initial  step,  we  believe,  is  to  impose  some 
formal  requirements  on  the  planning  process  for  pur¬ 
poses  of  establishing  the  basic  measurements  for  plan¬ 
ning  process  control.  More  specifically,  we  advocate 
formal  planning  models,  again  as  part  of  the  informa¬ 
tion  system.  It  must  be  granted  that  formal  planning 
docs  not  imply  formal  planning  process  control,  but  it 
is  the  point  of  departure  for  setting  up  the  necessary 
data  base  for  evaluation  of  performance.  Unless  there 
is  a  systematic  method  of  separating  the  assumptions 
and  forecasts  (the  model  and  its  parameters),  from  the 
logical  deductions  as  to  course  of  action  to  be  taken 
therefrom,  and  then  the  execution,  there  is  little  hope 
of  improvement  of  the  process.  Emery  (1965)  has 
waxed  fervently  and  at  length  on  this  subject.  Wc  agree 
with  him. 

Real-time  systems :  The  new  information  technology * 

Our  contention  is  that  the  new  computers  offer 
capabilities  that  enable  much  easier  construction  of 
intelligent  management  information  systems.  Both  the 
quality  of  the  data  base  and  the  power  of  the  pro¬ 
cedures  that  can  be  brought  to  bear  on  it  can  be  ma¬ 
terially  improved. 

Consider  first  the  now  generally  available  facility 
for  “on-line,  real-time”  data  processing  in  general  and 
real-time  operational  control  in  particular.  Since  real- 
time  processing  implies  up-to-the-minute  recording  of 
system-wide  individual  transactions  (status  changes), 
it  provides  uniquely  a  current,  global  data  base  (for 
operations).  In  other  words,  the  current  status  of  all 
operating  system  activities  is  known.  Furthermore,  since 
all  activities  are  “on-line”  to  the  central  processor, 

♦This  section  of  the  paper  is  a  partial  synopsis  of  Carroll 
(1966). 


access  to  large-scale  computational  power  can  be 
granted  in  order  to  respond  to  transactions  as  they 
arise.  This  has  distinct  implications  for  the  situation 
in  which  the  desired  response  is  in  the  form  of  con¬ 
trol  directions  which,  recall,  are  made  on  the  basis  of 
a  process  model. 

Time-sharing  is  a  product  of  the  same  technology, 
being  essentially  on-line,  real-time  computation  for 
multiple  users.  It  provides  for  man-machine  inter¬ 
action  in  problem  solving  without  creating  idle  time. 
The  close  coupling  thus  afforded  means  that  there 
exists  a  flexible  division  of  labor  between  man  and 
machine,  the  man  bringing  to  the  process  those  at¬ 
tributes  in  which  he  excels  in  close  cooperation  with 
the  superior  computational  powers  of  the  machine. 

Since  real-time  processing  and  time-sharing  are  based 
on  the  same  technology,  they  are  mutually  compatible 
and  both  are  compatible  with  conventional  “batch” 
processing,  it  has  been  noted  by  Carroll  (1966).  Conse¬ 
quently,  we  can  hypothesize  the  near-term  existence 
of  generalized  computers  which  possess  real-time  proc¬ 
essing  and  time-sharing  capabilities  in  which: 

.  .  .  whatever  permutation  or  combination  of 
human  and  machine  problem  solving  attributes 
is  needed  can  be  supplied  with  data  inputs  of 
whatever  quality  of  currency  or  scope  is  de¬ 
sired  (Carroll ,  1966,  p .  10). 

That  these  things  arc  good  in  general  is  undoubtedly 
true,  but  they  are  particularly  useful  for  the  pursuit  of 
intelligence,  we  assert.  In  operating  process  control, 
wc  have  noted  the  need  for  detailed  data  and  func¬ 
tional-temporal  association  thereof.  The  global  scope 
and  currency  of  the  data  in  these  generalized  systems 
meet  this  requirement.  Furthermore,  the  availability 
of  these  data,  coupled  with  the  computational  power 
in  real-time,  means  that  quite  complex  and  hence 
potentially  more  valid  process  models  can  be  utilized 
in  control.  And  finally,  wc  have  noted  that  inductive 
inference  is  a  faculty  limited,  for  the  moment  at  least, 
largely  to  human  beings,  yet  it  is  the  fundamental 
process  of  increased  understanding.  Through  the  new 
technology,  a  human  can  be  closely  coupled  to  op¬ 
erating  process  information;  he  can  monitor  the  process 
and  exercise  his  superior  capabilities  for  pattern  recog¬ 
nition,  abstraction,  hypothesis  formulation  and  test — 
in  short,  his  inductive  powers.  He  need  not  await  ac¬ 
cumulation  of  evidence;  he  is  on-line  to  the  operating 
process  even  though  it  may  be  geographically  dispersed. 
As  wc  will  observe  shortly,  this  testing  capability  is 
very  important.  Once  recognition,  reorientation  and 
understanding  of  the  process  is  established,  it  may  then 
be  introduced  into  the  system  itself  for  increased 
sophistication  and  intelligence. 

Some  of  these  points  can  be  illustrated  by  considera- 
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tion  of  the  application  of  real-time  processing  to  the 
job  shop  production  control  problems  cited  earlier. 
In  general,  the  simulation-based  schedules  were  noted 
as  providing  superior  control  because  they  provide  a 
more  valid  model  of  the  process  as  carried  out.  Ob¬ 
viously  if  the  model  is  valid,  departures  from  schedule 
mean  something.  But  this  is  true  for  only  a  short  period 
after  the  new  schedules  are  computed.  In  time,  minor 
deviations  from  schedule  accumulate,  machines  break 
down,  workers  arc  absent,  and  as  a  result,  the  model 
(i.e.,  the  schedule)  and  the  real  world  begin  to  di¬ 
verge.  This  is  the  “decay”  to  which  we  referred  pre¬ 
viously.  In  a  real-time  production  control  system,  de¬ 
tailed  decisions  on  product  movement,  relating  to 
sequencing  and  routing  of  jobs,  could  be  performed 
by  the  computer  (using  the  same  type  of  decision 
rules  employed  in  the  simulation).  But  because  the 
status  of  the  system  is  continually  updated,  the  decisions 
are  made  on  the  basis  of  true  current  status  as  op¬ 
posed  to  the  predicted  status  used  in  the  simulation 
approach  Consequently,  “decay”  would  not  occur  as 
easily  and  the  resulting  control  system  theoretically 
would  be  more  effective.*  Also,  we  would  have  a 
more  useful  system  for  diagnosis  because  deviations 
from  expected  behavior  would  more  likely  mean 
precisely  that  something  other  than  the  model 
is  wrong.  Analysis  of  causes  could  therefore  be  un¬ 
dertaken  without  risk  of  a  wild  goose  chase,  and  the 
fact  that  investigation  can  take  place  immediately 
provides  unparalleled  opportunity  for  accurate  recon¬ 
struction  of  the  “crime,”  it  is  conjectured.  But  good 
diagnosis  docs  not  stop  with  crime  reconstructions. 
Its  greatest  value  lies  in  its  educational  aspects.  The 
more  confident  one  is  of  cause  and  effect  relationships 
the  stronger  pattern  association  and  the  faster  the 
remedial  action. 

Diagnosis  is,  of  course,  often  a  more  subtle  proc¬ 
ess  than  is  implied  by  the  running  down  of  variations 
from  plan.  It  often  requires  “browsing”  through  his¬ 
torical  data,  classifying,  normalizing,  rearranging  and 
the  like.  The  flexible  interaction  feature  of  time-shar¬ 
ing  provides  great  convenience  and  power  for  so  do¬ 
ing.  Being  able  to  think  between  interactions  with  the 
computer  is  at  the  heart  of  the  concept  of  “man-com¬ 
puter  symbiosis”  advanced  by  Lieklidcr  (1965)  among 
others. 

The  advantages  of  this  new  technology  are  perhaps 

:!,The  relalive  effectiveness  of  real-time  versus  periodic  sched¬ 
uling  was  tested  by  Kogan  (1966)  In  the  cases  studied,  the 
theory  was  found  to  be  valid.  Of  course,  we  must  admit  that 
the  comparisons  are  influenced  extensively  by  the  expertise  of 
the  one  who  simulates.  Fven  so,  since  we  are  dealing  with 
non-deterministic  systems,  real-time  control  will  out  perform 
controls  based  on  fixed  theoretical  models. 


even  more  marked  in  the  planning  process  control 
domain.  First  of  all,  planning  itself  is  a  natural  man- 
machine  proeess,  it  has  been  frequently  noted  (Carroll, 
1965b;  Emery,  1965.  Simply  being  able  to  intertwine 
the  heuristically  well-endowed,  intuitive  and  subjective 
planner  with  his  model  offers  enormous  advantages. 
But  the  greater  advantage  may  come  in  the  metamodel¬ 
ing  process,  that  is,  modeling  the  planner’s  behavior 
for  purposes  of  ultimate  improvement  (assuming  his 
use  of  a  computer  model).  Capturing  the  planner's 
“trace,”  the  detailed  sequence  of  steps  he  takes  in 
arriving  at  a  decision,  is  quite  possible.  Given  the 
planner's  cooperation,  it  is  simply  a  question  of  ob¬ 
taining  the  hard  copy  transcript  of  his  session  with 
the  computer  model,  for  example.  And,  of  course,  the 
diagnostician  is  equipped  w'ith  a  unique  linkage  to  the 
process  he  is  attempting  to  understand. 

Exploitation  of  the  power  of  these  generalized  sys¬ 
tems  for  operating  process  control,  planning,  and  plan¬ 
ning  process  control  has  been  the  subject  of  research 
by  Carroll  (1965b)  and  colleagues  at  Project  MAC. 

Research  in  intelligent  information  systems 

When  we  view  where  we  stand  in  relationship  to 
our  goal  of  intelligence,  we  realize  just  howr  little  is 
known  about  the  techniques,  the  economies,  and, 
broadly,  the  phenomenon  of  intelligence. 

We  list  belowr  a  few'  areas  of  research  which  we  feel 
represent  promising  starting  points  for  improvement  in 
this  regard. 

Data  base  and  information  systems  for  intelligence 

One  of  the  prerequisites  for  the  implementation  of 
intelligent  systems  is  an  efficient  data  base.  For  this 
we  need  better  understanding  of  the  data  required  for 
improved  understanding.  We  have  stated  the  general 
specifications  for  detailed  and  asssoeiative  data;  but 
there  are  numerous  questions  begged  by  this,  such  as 
what  detail  and  what  means  for  association.  In  short, 
we  need  some  operational  specifications  (subjected 
to  economic  analysis  hopefully)  and  some  demon¬ 
strably  better  mousetraps  in  the  data  structure  domain 
There  is  a  dilemma  involved  in  the  question  of  what 
detail,  for  example.  One  simply  cannot  specify  a  priori 
what  detail  or  even  what  variables  to  measure  until  un¬ 
explained  differences,  problems  (or  successes)  occur, 
and  hypotheses  are  generated.  What  needs  to  be  es¬ 
tablished  is  the  usefulness,  for  diagnostic  purposes,  to 
provide  guidelines  on  collection  of  possibly  relevant 
data  (as  opposed  to  already  known  relevant  data).  In 
short,  some  theory  is  needed. 

Some  general  research  in  the  structure  of  the  as¬ 
sociative  data  bases  and  the  procedures  for  exploiting 
this  association  is  in  proeess  by  Zannetos  and  Sahin 
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(1966).  This  can  be  described  briefly  (and  specula¬ 
tively)  as  follows: 

The  data  base  requirements  for  an  associative  in¬ 
formation  system  will  be  mainly  the  same  as  those  of 
a  functional  accounting  system  previously  described, 
with  one  major  difference.  Instead  of  using  raw  data 
as  the  indecomposable  modules  for  storage,  manipula¬ 
tion,  and  causal  association,  patterns  or  configurations 
of  data  will  now  be  used.  The  faculties  of  understand¬ 
ing,  we  might  even  say  “consciousness,”  loom  into 
prominence  and  somehow  must  be  captured  and  in¬ 
corporated  in  the  data  base  by  means  of  these  patterns. 

To  get  at  the  question  of  procedure,  assume  that 
we  have  an  organization  with  well-established  objec¬ 
tives  and  a  dominant  (i.c.,  chosen)  plan  to  accomplish 
them.  Given  the  dominant  plan,  we  assume  that  the 
organization  will  be  able  to  specify  the  operations  that 
arc  necessary  to  achieve  its  objectives.  Now  for  each 
dominant  plan  there  must  be  a  given  configuration 
(pattern)  of  resource  utilization,  which  will  best  im¬ 
plement  the  dominant  plan,  at  least  on  an  a  priori 
basis.  With  the  dominant  resource  configuration  es¬ 
tablished,  a  dominance  ranking  of  these  resources  can 
be  made  in  turns  of  a  onc-dimcnsional  index.  Such 
ranking  may  be  in  terms  of  opportunity  costs  or  loss 
functions.  Wc  arc  only  interested  in  the  dominant 
plans,  resource  configurations,  and  resources,  and  in 
proximate  ordinal  rankings.  (The  hypotheses  which 
the  system  will  generate  and  the  search  which  will 
follow  for  testing,  all  of  which  arc  part  of  the  system, 
will  compensate  for  such  approximations.)  Further¬ 
more,  wc  are  only  interested  in  probabilistic  associa¬ 
tions. 

The  next  requirement  is  the  association  of  resources, 
at  the  point  of  acquisition,  with  the  various  (major) 
attributes  of  such  resources.  These  sets  of  attributes 
arc  given  a  temporal  index  and  also  contain  entries 
indicating  the  major  physical  characteristics  of  re¬ 
sources  (among  which  arc  cost  and  capacity  informa¬ 
tion).  The  attributes  of  resources  arc  ranked  once 
again  according  to  dominance  which  obviously  is  dic¬ 
tated  by  the  dominant  plan. 

Each  one  of  the  resources  in  the  dominant  configura¬ 
tion,  no  matter  what  its  ranking  therein,  may  be  the 
dominant  resource  in  another  configuration  of  sub¬ 
ordinate  resources,  as  well  as  non-dominant  member 
of  other  configurations.  By  means  of  this  “dominance” 
procedure  a  hierarchy  of  associations  both  vertical  and 
horizontal  is  established. 

With  the  above  as  a  brief  description  of  the  system, 
let  us  now  look  at  (diagnostic)  hypothesis  generation 
at  the  operating  level,  because  this  is  one  of  the  great¬ 
est  attributes  wc  wish  to  impart  to  the  information 
system.  The  signals  which  trigger  hypothesis  genera¬ 


tion,  arc  of  at  least  three  kinds.  They  may  originate 
in: 

•  The  difference  between  resource  utilization  (both 
quantities  and  attributes  used)  as  specified  in  the 
prior  dominant  plan  (model)  and  as  reflected  in 
operations. 

•  The  dominant  resource  configuration  of  a  pro¬ 
posed  plan,  if  it  docs  not  use  the  most  dominant 
characteristic  of  each  of  the  resources  proposed 
for  the  implementation  of  the  plan.  (If  the  new 
plan,  after  search,  is  still  found  to  be  dominant 
then  an  updating  of  the  resource-attribute  vec¬ 
tors  will  be  necessary  to  incorporate  the  latest 
ranking.) 

•  The  presence  of  “slack”  in  some  dominant  resource 
which  will  necessitate  a  change  in  the  opportunity 
cost  of  this  resource  and  a  temporary  change  in 
the  dominance  rank.  (The  system  scans  for  slack 
in  capacity  starting  from  the  most  dominant  re¬ 
source  downward.) 

Once  the  signal  is  received,  on  the  basis  of  its  content, 
the  system  immediately  associates  at  least  two  patterns 
with  it:  the  highest  hierarchical  pattern  where  the 
resource  appears,  whether  in  a  dominant  role  or  not, 
and  the  pattern  in  which  it  is  the  most  dominant  re¬ 
source.  Now  the  search  begins  for  term  by  term 
comparisons  of  the  prior  patterns  (plan)  and  those 
derived  from  operations  or  included  in  the  proposed 
plans,  and  hypotheses  arc  tested.  Depending  on  the 
results  of  these  tests  the  descriptive  sets  of  resource 
attributes  may  be  rearranged  and  assigned  new  tem¬ 
poral  designation  for  subsequent  reference.  Also,  the 
data  base  is  hierarchically  reorganized. 

In  addition  to  its  diagnostic  properties,  this  system 
also  holds  promise  for  providing  information  for  prog¬ 
nostic  purposes.  By  studying  the  intertemporal  changes 
in  the  resource-attribute  vectors,  the  system  may  now 
generate  hypotheses  and  test  for  the  existence  of  pat¬ 
terns  of  relationships  which  can  be  used  for  plan¬ 
ning  process  control  (and  also  assess  efficiency  of  the 
decisions  by  bringing  everything  back  to  the  point  of 
origin  and  thus  facilitate  learning).  For  this  latter  task 
wc  need  systems  with  man-machine  interaction  fea¬ 
tures. 

In  order  to  operate  efficiently,  a  system  such  as  the 
one  described  here  cannot  obviously  depend  on  “brute 
force”  or  exhaustive  sequential  search,  because  of  im¬ 
mense  combinatorial  problems.  Wc  suspect  that  we 
must  use,  therefore,  parallel  search  techniques  or  some 
hybrid  system.*  Also  parallel  processing  capabilities 
are  desirable  for  reasons  of  efficiency.  As  for  the  cues 

♦Selfridgc  and  Ncisscr  (1963)  have  commented  on  the  relative 
merits  of  the  two  search  strategies. 
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that  trigger  pattern  retrieval  and  association,  they  must 
not  refer  to  locations  of  stored  messages  but  to  the 
eonent.  Finally,  we  noticed  that  there  is  a  need  for 
some  hierarchical  organization  of  the  data  base  with 
distributed  logic.  This  relative  decentralization  allows 
flexibility  for  learning  and  self-organization,  but  also 
necessitates  functional  association  of  the  various 
modules  of  the  data  base.  The  problem  of  deciding 
how  much  logic  is  to  be  distributed  and  where  is  not 
an  easy  one  to  solve.  We  believe,  nonetheless,  that  it 
is  not  unlike  other  organizational  problems,  so  the 
theory  and  techniques  suggested  elsewhere  for  aiding 
in  the  design  and  evaluation  of  the  organization  struc¬ 
ture  are  also  applicable  in  this  ease  (Zannetos  and 
Carroll,  1966;  Zannetos,  1965a,  1965b). 

Theories  of  diagnosis  and  decision-making 

We  have  noted  that  the  “metamodeling”  problem  of 
planning  process  control  necessitates  modeling  the 
planning  process  itself.  But  planning  is  a  decision  proc¬ 
ess,  so  this  amounts  to  modeling  a  deeision-maker. 
This  has  been  an  active  area  of  research  for  some 
years  now,  notably  by  students  of  Simon  and  Newell 
such  as  Clarkson  (1962).  Other  approaches  have  been 
studied  by  Bowman  (1963)  and  his  students.  However, 
no  research  has  been  directed  to  modeling  the  type  of 
modeling  process  involved  in  planning  and  we  think 
this  would  be  useful. t 

Moreover,  diagnosis,  in  the  sense  that  we  have  em¬ 
ployed  the  term,  is  a  poorly  understood  process  at 
best.  There  is  much  work  going  on  in  medical  diag¬ 
nosis,  but  unfortunately  for  our  purpose  this  is  what 
I  might  be  called  “discriminatory  diagnosis,”  in 
which  the  relationship  of  symptoms  to  diseases  is  taken 
as  known  (probabilistically),  rather  than  “inductive 
diagnosis”  in  which  no  such  relationship  is  available. t 
However,  work  in  medical  diagnosis  will  undoubtedly 
provide  some  general  insights.  Particularly  promising 
is  the  research  of  Gorry  (n.d.)  who  is  attempting  to 
create  a  general,  diagnostic  model  (“general”  means 
environment  independent — applicable  to  sick  people, 
sick  cars,  sick  computer  programs).  His  emphasis  is 
on  discriminatory  diagnosis,  but  we  suspect  that  the 
data  structures  and  much  of  the  logic  of  his  procedure 
are  applicable  to  the  less  well-structured  inductive  diag¬ 
nostic  problem  as  well. 

In  addition  to  understanding  individual  decision 

tNewell  and  Simon  (1963)  did  incorporate  a  “planning” 
mechanism  in  lheir  “General  Problem  Solver,”  it  should  be 
noted,  but  lheir  type  of  planning  and  ours  are  only  generically 
related. 

{What  must  be  supplied  in  inductive  diagnosis  is  the  hypothesis 
of  relationship,  a  task  we  have  already  allocated  to  lhe  man 
in  lhe  man-machine  partnership. 


behavior,  we  have  noted  that  planning  is  often  a  group 
process;  it  is  performed  by  a  “team.”  Team  decision 
making  is  not  well  understood.  The  pioneering  work 
of  Radner  (1959);  Marshak  (1955)  and  Kriebel 
(1963)  brings  some  organization  to  this  area  and  the 
recent  work  of  Clarkson  (1966)  in  descriptive  (com¬ 
puter  simulation)  approaches  to  group  decision-mak¬ 
ing  is  directly  relevant  to  the  problem.  The  coming 
general  availability  of  time-sharing,  which  enables 
group  cooperative,  interactive  problem  solving  and 
monitoring,  should  greatly  facilitate  research  in  this 
area. 

SUMMARY  AND  CONCLUSIONS 

We  have  specified  some  features  of  information  sys¬ 
tems  which  are  capable  of  increasing  management's 
understanding  of  its  environment  and  its  rationality 
in  coping  with  it — its  intelligence.  In  so  doing,  we 
have  focused  our  attention  first  on  operating  process 
control,  in  which  intelligence  is  increased  by  causal 
induction,  diagnosis  of  the  factors  which  underlie 
process  behavior.  This  is  conveniently  framed  as  im¬ 
proving  the  model  of  the  process.  We  then  discussed 
the  higher  level  problem  of  planning  process  control 
which  we  noted  was  typical  of  higher  order  intelli¬ 
gence  problems.  This  too  involves  model  improvement, 
but  in  this  ease  it  is  improving  the  model  of  what  is  a 
modeling  process  itself. 

We  have  reduced  our  discussion  to  size  by  ignoring 
some  aspects  of  system  design.  For  example,  we  have 
ignored  the  general  dimension  of  information  availabili¬ 
ty.  There  exist  in  this  area  several  issues,  to  wit:  Should 
information  relating  to  detailed  performance  of  lower 
level  organizational  subunits  be  freely  accessible  to 
higher  level  managers?  To  what  extent  should  the 
opposite  take  place?  Should  parallel  organizational 
units  share  data  on  their  status  and  performance? 
These  are  real  issues  which  are  related  in  part  to  “man¬ 
agerial  style”  but  they  also  impinge  on  organizational 
intelligence.  The  “multiple  split  personality”  aspect  of 
organizational  behavior  is,  we  suspect,  intimately 
linked  to  the  question  of  dissemination  of  information. 

Another  area  that  we  have  ignored  encompasses  the 
perennial  issues  of  “cost  and  value”  of  the  generated 
information.  No  doubt,  trade  offs  must  be  established 
between  the  cost  of  the  system,  detail  and  purity  of  in¬ 
formation,  reality  of  representation  among  other  fea¬ 
tures,  and  the  objective  as  well  as  the  often  subjective 
utility  of  the  results.  All  these  issues  we  chose  to 
leave  outside  the  purview  of  this  presentation  for 
reasons  of  expediency  and  without  prejudice. 

Within  the  scope  of  the  general  problem  we  have 
attacked,  we  first  delineated  the  basic  processes  under¬ 
lying  control  systems  and  stressed  the  significance  of 
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the  diagnostic  process  in  establishing  eause  and  effect 
relationships.  Understanding  of  causal  dynamics  is 
important  to  us  not  so  much  for  its  regulative  power 
but  for  its  educative  role.  Learning  and  updating  of 
the  models  used  are  eritieal  managerial  functions  es¬ 
pecially  for  the  planner. 

Although  we  view  planning  and  control  as  two  hier¬ 
archical  processes  linked  together  hierarchically,  for 
purposes  of  exposition,  we  separated  planning  process 
eontrol  from  operating  process  control.  We  advocated 
eontrol  not  only  of  operations  but  also  of  the  planning 
process  itself  so  that  inefficiencies  do  not  creep  in- 
eipiently  through  the  fixities  of  planning. 

Formal  models  arc  prerequisite  before  any  type  of 
feedback  eontrol  proeess  takes  place.  While  there  is 
ample  evidence  of  their  use  for  operating  process  eon¬ 
trol  there  is  very  little  indication  that  they  are  used  for 
planning  proeess  control.  Planning,  therefore,  remains 
as  an  ad  hoc  proeess  mostly  outside  the  regular  in¬ 
formation  and  eontrol  system.  But  even  in  the  case  of 
operating  proeess  eontrol  we  found  by  examining  the 
state  of  the  art  that  the  models  used  are  mostly  naive 
and  capable  of  only  cursory  symptomatic  eontrol.  In¬ 
ductive  diagnosis  for  establishing  of  causalities  and 
changing  the  model  and  its  parameters  whenever  neces¬ 
sary  is  in  general  missing. 

In  order  to  improve  our  present  eontrol  systems 
for  operations  we  advocated  associative  data  bases  with 
pattern  recognition  and  pattern  generation  capabilities. 
Functional  relationships  (cause-effect),  probability  dis¬ 
tributions,  and  procedural  instructions  (to  be  followed 
upon  association)  must  be  part  of  the  data  base. 

For  planning  proeess  eontrol,  in  addition  to  the 
above  specifications,  we  argued  for  system  capabilities 
to  reeognize  patterns  in  cases  where  ambiguity  exists. 
This  will  give  the  manager  sufficient  lead  time  to 
prevent  undesirable  consequences,  and  also  update  the 
planning  model  itself.  If  the  remedial  action  specifica¬ 
tions  and  the  model  refinement  are  part  of  the  control 
process  then  such  a  system  we  ealled  prognostic. 

Finally  we  suggested  some  steps  for  the  realization 
of  intelligent  information  systems.  In  the  area  of  op¬ 
erating  proeess  control  wc  believe  that  wc  have  made 
enough  progress  and  also  have  the  technology  (real- 
time  systems)  to  improve  significantly  the  intelligence 
of  our  present  control  systems.  As  for  planning  process 
control,  wc  only  speculated  on  the  basis  of  our  on¬ 
going  research  designs  and  suggested  a  procedure  for 
creating  dominant  patterns  whieh  will  allow  associa¬ 
tion  and  automatic  hypotheses  generation  and  testing. 
In  addition  to  more  progress  in  conceptual  system 
design,  theories  of  diagnosis  and  decision  making,  wc 
sec  also  the  desirability  for  hardware  and  software 
which  will  allow  associations  on  the  basis  of  content 


and  parallel  scareh,  parallel  processing. 

In  conclusion,  we  believe  that  increased  organiza¬ 
tional  intelligence  is  possible  and  greatly  to  be  facili¬ 
tated  by  new  advances  in  information  technology.  Per¬ 
haps  the  greatest  progress  ean  be  made,  however,  by 
simply  recognizing  that  increasing  intelligence  is  a  legit¬ 
imate  goal  for  information  systems  design,  and  that 
there  are  some  straightforward  steps  whieh  ean  be 
taken  towards  that  goal. 
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Guidelines  for  simulation  model  development 
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INTRODUCTION 

This  paper  proposes  a  set  of  guidelines  for  developing 
a  simulation  model  for  planning.  The  guidelines 
were  formulated  and  tested  in  the  process  of  develop¬ 
ing  several  industrial  simulations.  One  conclusion  of 
this  experience  which  seems  acceptable  to  most  in¬ 
dividuals  involved  in  creating  analytical  abstractions, 
is  that  better  abstractions  result  when  the  decision 
makers  assume  an  active  role  in  the  development  of 
the  abstraction.  These  suggested  guidelines  have  facil¬ 
itated  the  involvement  of  the  decision  maker  in  model 
development  in  addition  to  being  useful  in  the  develop¬ 
ment  of  planning  models.  The  reasons  for  proposing 
them  at  this  time  is  to  focus  attention  on  the  importance 
of  utilizing  the  decision  maker’s  concept  of  his  en¬ 
vironment,  and  the  necessity  that  all  individuals  con¬ 
cerned  with  the  construction  of  a  simulation  model 
understand  the  developmental  nature  of  the  project. 

Our  guidelines  for  model  building  are  defined  as  the 
attitudes,  policies  and  postulates  followed  by  those 
involved  during  the  design  and  development  of  an  ab¬ 
stract  representation  of  an  environment.  A  premise  of 
the  guidelines  is  that  better  models  are  the  result  of  a 
joint  developmental  effort  of  the  individuals  who  are  to 
use  the  model  as  a  tool  (the  planners)  and  the  in¬ 
dividuals  who  arc  designing  and  programming  it  (the 
modelers).  The  guidelines  arc  as  follows: 

•  The  simulation  model  is  conducted  as  a  develop¬ 
mental  project  to  aid  the  planning  process. 

•  An  important  characteristic  of  the  project  is  the 
evolution  of  goals,  uses,  and  specifications  of  the 
model  as  it  relates  to  the  planning  process. 

•  The  planner’s  intuition  is  the  most  appropriate 
reference  of  the  pertinent  environment;  therefore, 
it  is  the  role  of  the  model  to  improve  the  con¬ 
sistency  of  the  planner's  intuition  and  to  make 
them  aware  of  new  information  requirements. 

•  A  prime  function  of  the  model  is  to  amplify  the 
intuition  of  the  planner  by  generating  a  spectrum 
of  analyses  for  certain  codifiablc  conditions. 

There  were  other  guidelines  we  followed  in  the 


development  of  planning  models,  but  they  either  seemed 
unique  to  the  situation  or  too  ambiguous  to  sensibly 
defend.  An  attitude  which  is  essential  to  all  successful 
model  building  is  the  eventual  faith  in  the  model  as 
a  tentative  statement  of  causal  factors  influencing  the 
resources  to  be  allocated.  T  here  are  several  published 
sermons  on  the  necessity  of  the  belief  in  the  order  of 
nature,  and  we  will  not  dwell  on  the  subject  (Ackoff, 
1962;  Buzzcll,  1964).  Perhaps  the  above  guidelines  can 
be  considered  a  few  of  the  necessary  conditions  to 
engender  a  planner’s  conviction. 

A  discussion  of  the  guidelines 

The  sole  criterion  of  success  in  our  discussion  of 
simulation  models  and  planners  is  that  the  planner 
utilize  the  model  to  improve  his  understanding  of  the 
world  in  order  to  allocate  resources.  An  unused  model, 
no  matter  how  elegant,  is  a  failure.  It  may  be  that  the 
significant  “use”  a  planner  makes  of  a  simulation  model 
is  in  its  development  to  refine  his  own  concept  of  his 
problem.  The  discussion  below  deals  with  how  the 
guidelines  serve  to  encourage  the  planner  to  contribute 
to  the  development  of  a  model  and  what  the  modeler 
should  have  in  mind  to  facilitate  this  contribution. 

The  involvement  of  pertinent  individuals  in  the  de¬ 
velopmental  process  of  a  model  is  critical  for  two 
reasons:  ( 1 )  to  generate  and  distill  an  appropriate  data 
base  from  which  a  pertinent  model  can  be  created;  (2) 
to  develop  a  commitment  to  the  model  as  a  professional 
managerial  goal.  The  first  reason  is  obvious  and  would 
probably  not  require  a  great  deal  of  the  planner’s  time. 
However,  developing  a  feeling  of  commitment  is  just 
as  critical  as  “truth”  and  requires  an  adequate  incuba¬ 
tion  period  for  modelers  and  planners  alike. 

The  planners  and  modelers  should  individually  con¬ 
sider  the  simulation  project  as  a  means  to  increasing 
their  understanding  of  the  operation  of  the  firm  and 
thus  their  influence  on  the  operation.  To  gain  the  most 
insight  from  the  project  each  individual  should  work 
to  improve  the  effectiveness  of  the  model  as  it  relates 
to  his  activities. 
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It  is  critical  that  all  pertinent  personages  understand 
the  functioning  of  the  model  such  that  they  feel  a  need 
to  participate  in  defining  aspects  of  the  model  germane 
to  their  sphere  of  influence  and  in  responding  critically 
about  the  formulation  of  the  simulation.  Given  ade¬ 
quate  participation  the  model  can  become  a  unique  re¬ 
source  in  that  it  represents  the  firm  as  an  entity  with 
few  organizational  boundries.  This  seems  to  generate 
a  depersonalized  professional  attitude  in  the  planner 
toward  the  model,  which  allows  the  model  to  be  scru¬ 
tinized  for  appropriateness  in  painful  detail  without 
bruising  individual  egos.  A  successful  taetie  to  obtain 
this  complete  involvement  is  to  have  the  simulation 
project  a  responsibility  of  the  top  planning  executive. 

As  is  obvious,  having  the  boss  head  the  project  is 
but  one  step  toward  creating  a  decent  simulation.  It 
is  necessary  that  all  individuals  concerned  have  a 
thorough  understanding  of  the  nature  and  potential  of 
a  simulation  project.  It  is  suggested  that  by  constituting 
the  project  as  a  research  and  development  venture  on 
the  managing  proeess  an  orientation  is  provided  to  the 
concerned  individuals  in  which  all  are  expected  to  learn. 
This  learning  attitude  is  helpful  when  exploring  how 
one  ean  formulate  heretofore  non-explicit  relations. 
In  addition  a  developmental  projeet  by  nature  should 
commit  a  management  to  a  sizeable  budget  over  an 
extended  period  of  time.  The  results  of  this  expenditure 
arc  uncertain  and  therefore  the  projeet  should  regularly 
be  appraised  as  to  its  effectiveness.  This  appraisal 
proeess  is  especially  important  in  regard  to  simulations 
intended  to  aid  the  planning  proeess. 

The  definition  of  criteria  to  evaluate  improvement 
of  the  planning  proeess  is  a  difficult  art  and  requires 
experimentation  and  attention.  However,  foeusing  on 
this  aspect  of  the  model’s  impact  provides  an  appro¬ 
priate  perspective  for  considering  the  effectiveness  of 
the  model.  The  appraisal  should  allow  adequate  elapsed 
time  for  the  development  of  a  series  of  plans  con¬ 
current  with  the  implementation  of  the  model.  During 
this  time  the  defense  for  continuing  financial  support 
for  the  model  probably  rests  on  the  degree  to  which 
it  stimulates  the  management  to  consider  their  planning 
proeess.  After  the  model  is  being  utilized  as  an  active 
aid,  support  should  be  judged  on  documental  evidence 
produced  by  key  planners.  The  model  should  not  be 
judged  solely  on  appropriateness  of  results,  number 
of  plans  evaluated,  or  mechanics  of  operation.  These 
ean  be  modified  by  utilizing  different  resources  to  de¬ 
velop  the  model.  It  should  be  judged  on  how  effective 
it  is  in  improving  the  planning  proeess.  In  essence  a 
decent  model  seems  to  require  a  budget  for  an  extend¬ 
ed  time  during  which  periodic  reviews  are  made  of 
the  decision  proeess,  not  model  details. 


The  evolutionary  nature  of  planning  models 

The  prime  reason  for  a  pre-ordained  extended  life 
of  a  planning  model  projeet  is  that  the  only  constant 
characteristic  of  a  simulation  model  is  change.  The 
product  of  this  evolutionary  proeess  is  assisted  if  the 
changing  nature  of  the  model  is  understood  by  all  as¬ 
sociated  with  the  model  from  the  very  start  of  the 
projeet.  Models  with  a  tradition  of  change  will  en¬ 
courage  the  planners  to  attempt  to  define  hazy  ideas 
and  to  experiment  with  formulating  relationships,  as 
conjectures  ean  be  changed  if  desired.  It  will  also 
induce  the  modelers  to  design  their  procedures  to  ac¬ 
commodate  changing  definitions  and  specifications. 

The  most  important  reason  for  designing  an  adapt¬ 
able  simulation  model  development  is  the  very  survival 
of  the  simulation.  An  adaptable  and  changing  model 
is  essential  if  the  model  is  to  be  used  over  an  extended 
period  of  time.  Assuming  the  model  is  to  operate  as  an 
agent  for  improving  the  planning  proeess,  its  main 
function  may  well  be  as  a  stimulus  to  search  for  a 
definition  of  what  the  planner  has  not  included  in  the 
model.  This  improvement  proeess  seems  to  be  one  of 
continuous  redefinition  of  the  planner’s  concept  of  the 
pertinent  forces  in  the  environment  and  growth  in  the 
modeler’s  ability  to  adequately  represent  these  forces. 
The  model  must  continuously  reflect  the  planner’s 
improved  concepts  or  fall  into  disuse  by  the  decision 
makers.  A  successful  simulation  project  for  planning 
will  stimulate  the  continuous  growth  of  the  participants 
as  evidenced  by  an  improving  model. 

The  reference  base  utilized  for  the  initial  stages  of 
model  development  is  critical  to  the  goal  of  generating 
a  useful  tool.  It  is  suggested  the  most  appropriate  ref¬ 
erence  is  the  planner’s  definition  of  the  pertinent  in¬ 
fluences  in  the  environment  and  how  he  thinks  they 
relate  to  each  other. 

A  planner  with  significant  budgetary  responsibility 
is  assumed  to  have  had  an  adequate  involvement  with 
his  environment  to  have  developed  an  understanding 
of  what  elements  are  critical  to  the  success  of  his  opera¬ 
tion.  His  measures  of  these  elements  often  range  from 
precise  dollar  figures  to  vague  intuitive  impressions, 
but  all  are  important  and  real  to  him.  It  seems  reason¬ 
able  to  accept  the  planner’s  notion  of  his  business  as 
fact  and  to  attempt  to  substantiate  his  concept  by 
programming  a  model  to  imitate  the  concept.  Normally 
it  is  impossible  to  explicitly  define  all  of  the  factors  a 
planner  considers.  In  addition  individual  planners  will 
not  be  consistent  with  each  other  or  emphasize  the 
same  aspects  of  the  environment.  To  cope  with  these 
concomitant  ambiguities,  the  model  should  be  pro¬ 
grammed  to  eodify  as  many  factors  as  possible  with 
freedom  for  the  planner  to  modify  the  impact  and 
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range  of  each  variable.  Where  variables  cannot  be 
defined,  provision  should  be  made  for  direct  planner 
influence  as  he  sees  fit.  The  goal  of  the  modeler  is  to 
generate  an  abstraction  that  adapts  comfortably  to  how 
the  planner  considers  his  resources  allocation  problem. 
The  process  of  programming  the  well-defined  variables 
should  involve  the  planners  and  modeler  in  order  that 
both: 

•  Evaluate  the  sensitivity  of  the  environment  to 
change  in  the  selected  variables. 

•  Discover  new  methods  of  combining  variables. 

•  Resolve  inconsistencies  or  ambiguities  between 
planners  and  the  environment. 

This  latter  process  often  serves  as  a  basis  for  data 
collection  to  define  missing  relations  or  to  test  in  the 
environment  the  validity  of  assumed  relationships.  A 
clear  definition  of  how  the  simulation  functions  is  es¬ 
sential  for  the  mutual  consideration  of  relationships 
that  govern  the  operation  of  the  simulation. 

An  overt  goal  of  most  simulation  projects  for  plan¬ 
ning  is  that  operationally  the  simulation  model  is  to 
serve  as  an  analytical  tool  for  the  planner.  To  serve 
effectively  it  must  be  formulated  to  produce  results 
which  are  compatible  with  the  planning  procedures. 
Elegant  reports  can  be  generated,  but  care  should  be 
taken  to  well  define  the  limits  as  well  as  the  potential 
power  of  the  model  as  an  analytical  engine.  The  model¬ 
er  and  planner  should  continuously  appraise  what  is 
more  economical  and  effective  for  the  model  to  ac¬ 
complish  versus  the  planner.  At  this  point  in  time  it 
does  not  seem  economically  feasible  to  model  complete¬ 
ly  an  environment  as  effectively  as  a  good  human  desi- 
ion  maker.  However,  a  simulation  model  can  perform 
quickly  and  accurately  a  long  involved  sequence  of 
well  specified  events  to  produce  an  answer  in  pre¬ 
defined  terms.  How  the  planner  will  use  these  answers 
is  important  in  the  development  of  the  model  and 
should  be  considered  at  each  step  of  the  program.  The 
goal  of  the  model  developer  is  to  develop  a  model 
Which  can  amplify  the  planner's  insight  to  a  resource 
allocation  problem.  At  present  the  method  of  am¬ 
plification  seems  to  be  a  prompt  evaluation  of  a  variety 
of  plans  under  a  range  of  assumed  conditions  which 
the  planner  defines. 

Using  the  guidelines  to  develop  a  planning  model 

How  these  guidelines  might  aid  the  development  of  a 
simulation  model  is  presented  in  the  following  synopsis 
of  a  simulation  project  in  an  industrial  firm.  The  firm, 
a  consumer  goods  producer,  with  sales  in  excess  of 
$200  million  was  planning  to  enter  the  European 
market.  This  was  its  first  venture  overseas  and  the 
executives  felt  a  need  for  improved  decision-making 
procedures  to  cope  with  the  unknown  seemingly  more 


complex  situation.  The  staff  aides  to  the  executive 
committee  had  suggested  a  simulation  model  might 
assist  the  planners  in  allocating  capital  overseas  to 
assure  an  orderly  and  profitable  entry  into  the  new 
market. 

A  scries  of  seminars  conducted  by  the  staff  with 
outside  consultants  was  initiated  to  explore  methods 
of  planning  in  general  and  the  potential  of  simulation 
models  in  particular.  A  topic  of  one  of  the  seminars 
was  the  evolutionary  nature  of  simulation  models  with 
the  expectant  change  in  understanding  of  their  en¬ 
vironment  by  the  individuals  using  the  model.  This 
session  produced  a  lively  interest  in  developing  a  model 
tailored  to  the  needs  of  the  executive  committee.  The 
eventual  result  was  a  tentative  five-year  program  for 
the  improvement  of  the  firms  strategic  planning.5  A 
three-year  capital  budget  of  approximately  $70,000 
per  year  was  allocated  to  the  development  of  a  simu¬ 
lation  model  which  was  to  be  defined  by  the  executive 
committee.  Progress  review  periods  were  to  be  held 
every  six  months  by  the  executive  vice  president  of 
sales  who  was  responsible  for  long-range  planning.  All 
vice  presidents  of  the  firm  and  members  of  the  executive 
committee  were  active  members  of  the  project  and 
agreed  to  spend  up  to  four  hours  per  week  to  de¬ 
velop  improved  planning  procedures. 

The  initial  proposal  developed  in  the  seminar  called 
for  a  tentative  model  to  be  operational  in  one  calendar 
year  at  which  time  a  redefinition  of  project  goals  and 
model  specifications  would  take  place.  At  the  con¬ 
clusion  of  the  seminars  the  management  concluded 
that  the  development  of  a  simulation  model  for  plan¬ 
ning  seemed  to  be  a  viable  method  of  improving  their 
planning  process  and  could  provide  an  analytical  tool 
which  could  aid  them  considerably.  The  planning 
problem  was  to  allocate  probable  capital  resources 
over  an  extended  time  in  such  a  manner  as  to  most 
effectively  enter  and  become  established  in  the  Euro¬ 
pean  market. 

The  initial  model,  as  specified  by  the  executive 
committee,  called  for  a  representation  of  the  necessary 
resources  measured  in  dollars  and  elapsed  time  to 
produce  given  quantities  of  goods  and  the  demand 
generated  by  the  world  environment  for  the  company's 
products  by  price  and  type  of  product.  It  was  the  re¬ 
sponsibility  of  the  planners  to  identify  probable  gross 
changes  in  the  environment  such  as  new  models  or 
competitive  actions  and  to  define  possible  responses 
in  the  use  of  the  firm  resources  as  inputs  to  the  simu¬ 
lation.  The  model  would  then  give  a  profit  measure 
for  each  suggested  response.  The  planners  in  this 
project  were  the  vice  presidents  of  the  corporation 
responsible  for  the  Sales,  Manufacturing,  and  Research 
and  Development.  There  were  six  active  model  builders 
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of  heterogeneous  backgrounds.  The  group  included  a 
market  researcher,  economist,  operations  researeher, 
financial  aeeountant,  experienced  staff  planner  and  an 
analyst  programmer. 

The  modelers  began  the  projeet  by  attempting  to 
codify  what  the  planners  felt  were  important  influences 
on  their  planning  decisions  and  what  questions  they 
wanted  evaluated  by  the  model.  To  accomplish  this  the 
first  six  months  of  the  projeet  included  a  series  of 
meetings  with  each  planner  to  define  what  he  con¬ 
sidered  the  critical  aspects  of  the  firm’s  environment 
and  how  they  might  be  formulated  in  the  simulation 
model.  During  the  time  these  meetings  were  being 
held,  a  series  of  definition  papers  on  how  the  relation¬ 
ships  defined  by  the  planners  might  be  combined,  what 
data  would  be  required  and  what  reports  would  be 
generated  were  circulated.  These  papers  served  as  a 
basis  for  seminars  with  all  modelers  and  planners  to 
diseuss  what  should  be  included  in  the  simulation  model 
to  improve  strategic  planning.  After  the  seminars 
small  groups  would  often  discuss  a  specific  aspect  such 
as  how  a  new  product  should  be  represented  in  the 
production  proeess.  The  seminars  were  followed  by 
additional  two-  to  three-hour  individual  planner  con¬ 
ferences  with  two  or  three  members  of  the  group  to 
insure  the  planner’s  ideas  were  accurately  represented 
in  the  model  and  to  explain  how  the  model  was  func¬ 
tioning  with  other  planners’  definitions. 

A  continual  effort  was  made  by  the  modelers  to  de¬ 
fine  all  terms  as  clearly  as  possible  for  inclusion  in  a 
glossary  that  all  partie;pants  received.  The  glossary 
established  a  common  understanding  on  words  all  too 
often  not  well-defined  such  as:  assumptions,  sensi¬ 
tivity,  programming,  and  planning  horizon.  The  glossary 
aided  in  educating  the  planner  in  a  bit  of  modeling 
jargon  and  preventing  the  modelers  from  using  terms 
without  defining  them.  It  was  invaluable  in  documen¬ 
tation  of  the  model. 

Concurrent  with  the  planner  conference,  data  were 
eolleeted  as  specified  by  the  planners  to  define  the  re¬ 
sources  of  the  organization  in  a  manipulatable  fashion 
for  planning  purposes.  This  required  the  modelers  to 
work  with  the  staff  assistants  of  the  planners  in  an 
analysis  of  present  measures  of  what  the  planners  felt 
to  be  important.  The  available  aeeounting  data  did  not 
prove  sufficient  and,  therefore,  new  information  had  to 
be  created  and  stated  in  accessible  format.  For  example, 
data  were  collected  and  distilled  to  develop  a  pro¬ 
ductive  capacity  model  which  related  the  total  cost  and 
elapsed  time  of  producing  a  given  quantity  of  product 
to  the  mix  of  products  the  level  of  production.  The 
elapsed  time  required  to  aequire  additional  productive 
capacity  or  change  produet  mix  was  defined  in  ac¬ 


cordance  with  how  the  manufacturing  planner  thought 
the  eapaeity  responded. 

After  six  months  of  discussion  and  three  months  of 
data  collection  to  formulate  the  planners’  eoneept  into 
a  model,  the  programming  of  the  model  for  computer 
manipulation  was  started.  Simultaneous  with  our  pro¬ 
gramming  effort  a  seeond  series  of  meetings  were  held 
with  the  planners  on  how  they  might  utilize  the  simula¬ 
tion  in  their  on-going  planning  procedures.  It  was  felt 
important  to  maintain  the  planners’  interest  in  model  de¬ 
velopment  and  it  was  eonjeetured  that  during  the 
programming  proeess  a  few  revisions  in  the  planners’ 
model  would  be  neeessary.  The  individual  meetings 
soon  beeame  formalized  into  bi-monthly  planning  meet¬ 
ings  to  diseuss  the  state  of  the  model  and  how  it  might 
be  used  to  evaluate  alternative  resource  allocations, 
being  considered  for  future  two  year  periods.  These 
discussions  aided  the  modelers  in  defining  appropriate 
time  units,  ranges  of  aeeuraey,  specific  output  require¬ 
ments  and  potential  changes  in  the  input  variables. 
They  served  to  keep  the  planners  informed  on  the 
state  of  the  model  and  its  limitations. 

As  the  model  entered  the  final  debugging  stage  the 
meetings  focused  more  onto  methods  of  testing  the 
model  for  validity  and  formulating  plans  for  evalua¬ 
tion  by  the  model.  In  these  later  meetings  the  planners 
began  to  develop  expertise  in  explieity  defining  a 
feasible  range  of  eireumstanees  whieh  could  be  tested 
on  the  model.  This  in  turn  caused  the  modelers  to 
improve  the  model’s  ability  to  accurately  represent  a 
set  of  conditions.  The  results  of  this  iterative  proeess 
was  an  awareness  of  the  importance  of  experimental 
design  and  new  insight  to  the  evolutionary  aspect  of 
the  simulation  projeet.  Most  individuals  were  convinced 
it  was  a  rewarding  experience. 

The  strong  commitment  was  fortunate  as  early 
simulation  runs  proved  to  generate  quantities  of  useless 
output.  The  first  simulations  were  intended  to  repre¬ 
sent  10  years  of  sales  experience  in  the  international 
market.  The  simulations  on  the  average  produced 
rather  bizarre  sales  figures  and  production  demands. 
The  one  bright  side  was  that  the  cash  flows  resulting 
from  the  sales  were  consistent  with  past  experience. 
They  rediscovered  that  the  proeess  of  predicting  dynam¬ 
ic  demand  factors  and  economic  consequences  for  a  10- 
year  period  in  a  fairly  eodifiablc  fashion  allows  errors 
to  accumulate  to  bias  all  results.  The  planners  were 
not  dismayed  and  suggested  procedures  which  the 
modelers  could  incorporate  whieh  would  aid  in  the 
understanding  of  simulation  results.  Typical  error  pre¬ 
vention  procedures  called  for  the  planners  to  estimate 
for  the  next  two,  five,  and  eight  years  feasible  product 
priee  ranges,  and  estimates  of  production  eapaeity, 
given  the  present  base  of  the  company.  These  estimates 
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served  as  minimum  and  maximum  limits  on  capacity 
and  sales.  The  model  operated  within  these  bounds 
to  evaluate  the  proposed  price  structure,  time  of  product 
introduction  and  other  aspects  of  their  plan.  They  then 
considered  the  output  of  the  simulation  in  terms  of 
these  limits.  If  the  output  indicated  the  simulation  re¬ 
sults  hit  an  upper  limit  and  remained  there,  the  plan¬ 
ners  discounted  the  answer,  because  of  model  de¬ 
ficiencies  but  would  judge  that  the  plan  might  be  a 
better  one  than  a  plan  which  drove  the  model  to  the 
lower  limits.  These  procedures  have  afforded  a  basis 
for  jointly  testing  plans  and  their  assumptions  while 
evaluating  the  sensitivity  of  the  simulation  model  to  a 
variety  of  inputs  in  order  to  invesitgate  the  model's 
validity. 

Present  stage  of  simulation  utilization 

There  has  been  an  obvious  growth  in  the  attitude 
of  the  modelers  and  managers  as  to  what  should  be 
in  the  model  and  what  should  be  excluded.  A  few  of 
the  original  factors  included  as  determinants  of 
available  resources  of  influences  on  demand  have  been 
tested  and  found  unimportant.  But  of  more  interest 
is  the  number  of  new  factors  that  seem  to  be 
more  basic  and  casual  nature  than  our  original 
factors.  Originally  population  had  been  considered  as  a 
basic  variable  of  demand.  They  arc  now  considering 
age  distribution,  wealth  distribution,  geographical  dis¬ 
tribution,  and  other  factors  of  the  economy  in  a  given 
country  to  consider  its  market  potential.  Many  of  these 
factors  arc  still  being  tested  to  evaluate  whether  they 
arc  significant  in  the  long  run  and  probably  some  will 
be  discarded  Continual  evaluation  of  factors  in  the 
model  including  the  definition  of  assumptions  and 
defense  or  explanation  of  these  assumptions  is  now 
accepted  by  modelers  and  planners  alike.  Finally 
measures  specified  at  the  start  have  been  superseded 
by  new  ones.  Specific  dollar  requirements  and  time 
specifications  originally  desired  as  outputs  have  been 
replaced  by  requirements  of  rate  of  market  penetra¬ 
tion  or  equity  growth.  In  general  most  measures  of 
performance  arc  more  sophisticated  than  when  the 
project  began. 

The  planners  seem  to  be  evaluating  alternative  plans 
with  the  model  to  support  their  intuition.  They  sug¬ 
gest  that  the  model  has  improved  their  judgment  by 
testing  some  variables  which  heretofore  were  thought 
very  important  and  found  wanting  as  indicators  of 


future  significant  environmental  forces.  The  model 
development  in  part  has  forced  the  planners  to  define 
their  time  assumptions  explicitly  and  to  codify  cost 
assumptions  to  accommodate  manipulation.  This  has 
resulted  in  part  of  the  accounting  system  changing  to 
accommodate  an  evaluation  of  plans  rather  than  a 
reporting  of  the  accumulated  costs  of  past  activities. 
This  change  has  improved  the  firms  planning  pro¬ 
cedures  and  given  a  better  data  base  for  developing 
an  improved  model. 

At  present  the  model  can  almost  be  considered  a 
professional  goal  for  the  management  of  the  company 
as  they  are  committed  to  its  future  development.  They 
do  not  rely  upon  it  for  specific  decisions,  but  seem  to 
feel  it  a  useful  tool  for  improving  their  planning  pro¬ 
cedures.  Perhaps  at  some  future  date  they  will  rely  upon 
it  as  a  partner  in  decision  making  as  well  as  process 
improvement. 

CONCLUSION 

Most  modelers  are  aware  of  the  importance  of  involving 
the  planners  in  the  development  of  the  simulation 
model  to  achieve  an  operational  model.  We  have  had 
a  reasonable  degree  of  success  in  integrating  the  plan¬ 
ners  in  the  project  by  using  the  planner’s  concept  of 
his  environment  as  the  point  of  departure  for  the  model 
and  stressing  the  importance  of  change  from  the  begin¬ 
ning.  An  impediment  to  more  adequate  rapport  be¬ 
tween  modeler  and  planner  is  the  state  of  present 
computer  languages.  The  language  of  the  program  for 
the  model  has  to  be  interpreted  to  the  planner.  This 
interpretation  creates  ambiguities  and  misunderstand¬ 
ings  which  limit  the  effectiveness  of  present  simula¬ 
tions  as  a  tool  for  most  planners.  Hopefully  new  com¬ 
puter  languages  will  allow  the  modeler  and  planner  to 
conceptualize  the  simulation  model  in  the  language  it 
is  to  be  programmed. 
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by  I  RANK  H.  Eldridce 
Executive  Office  of  the  President 
Washington,  D.C. 

INTRODUCTION 

In  the  past  four  or  five  years  numerous  attempts  have 
been  made  to  apply  systems  analysis  techniques  to 
problems  of  command  control  and  communications 
(C‘)  in  the  Department  of  Defense.  Some  of  these  have, 
it  is  generally  acknowledged,  been  more  successful 
than  others. 

One  of  the  principal  problems  in  the  early  1960’s 
was  the  lack  of  an  adequate  resource  structure  for  C\ 
Hard  work  since  then  has,  to  a  large  extent,  solved 
this  problem  Physical  assets,  total  obligational  author¬ 
ity  and  manpower  have  been  specified  in  great  detail 
for  about  150  command  control  and  communications 
programs,  representing  the  entire  effort  of  the  De¬ 
partment  of  Defense  in  this  area.  Having  accomplished 
this,  studies  of  tradeoffs  and  alternatives  can  start 
from  a  common  data  base — a  necessary  prerequisite 
for  useful  cost  and  effectiveness  analyses. 

1  he  definition  of  C1  mission  objectives  has,  also, 
been  further  clarified  in  the  past  few  years.  By  the 
early  1960’s  the  role  of  C3  in  nuclear  warfare  had 
received  considerable  attention.  For  instance,  a  num¬ 
ber  of  models  had  been  constructed  showing  the  ef¬ 
fects  on  mission  objectives  of  various  types  of  thermo¬ 
nuclear  attacks  leveled  against  military  and  civilian 
headquarters  as  well  as  radio  and  I  and  line  communica¬ 
tions.  From  this  emerged  a  better  understanding  of 
the  use  of  mixed  systems  for  survival  and  the  need  for 
more  rapid  and  integrated  intelligence  to  meet  the 
needs  of  different  levels  of  command  under  various 
conditions  of  thermonuclear  attack.1,3 

More  recently  a  great  deal  of  actual  experience  has 
been  gained  in  the  use  of  command  control  and  com¬ 
munications  systems  in  crisis  situations  where  the  in¬ 
tent  is  to  achieve  U.S.  political  objectives  with  the 
minimum  possible  amount  of  conflict  escalation.  Leb¬ 
anon,  Congo,  Cuba,  Panama,  Santo  Domingo  have 
all  provided  opportunities  to  test  our  evolving  systems 
and  operational  methods. 

Many  of  the  systems  analysis  methods  developed 
by  the  Department  of  Defense  for  command  control 


and  communications  are  now  being  adapted  to  informa¬ 
tion  system  problems  of  other  Federal  Departments 
and  Agencies.  1  will  define  information  systems,  here, 
as  any  network  of  facilities  that  acquires  and/or  pro¬ 
cesses  data  for  use  in  controlling  resources.  In  this 
sense  an  information  system  can  be  either  purely  auto¬ 
matic  or  it  can  contain  manual  elements  for  monitor¬ 
ing,  display,  control,  or  other  command  functions. 

Associated  with  every  decision  are  both  ponderable 
issues  and  imponderable  ones.  The  analyst  should  con¬ 
centrate  on  the  ponderable  ones.  The  decision  maker 
must  consider  both. 

There  are  several  alternative  approaches  open  to 
the  systems  analyst  who,  once  having  identified  the 
principal  issues,  undertakes  to  address  them.  He  can 
collect  and  analyze  raw  data  or  he  can  build  models 
of  the  system  and  synthesize  data  that  will  serve  to 
answer  the  questions  or  he  can  do  both.  The  analyst 
is  advised  to  look  at  both  the  hardware  and  the  soft¬ 
ware,  or  operational  aspects  of  the  system,  in  order 
to  develop  a  balanced  context  for  his  study.  He  should 
relate  the  objectives  and  the  missions  of  a  support 
system,  such  as  telecommunications,  to  the  objectives 
and  missions  of  the  system  that  it  supports. 

The  choice  of  suitable  measures  of  effectiveness  is 
a  critical  part  of  the  analysis.  He  should  be  aware  of 
what  it  should  do.  By  choosing  measures  of  effective¬ 
ness  that  are  related  to  the  missions  of  the  system 
supported  he  can  avoid  selecting  effectiveness  criteria 
that  relate  to  less  important  side  issues. 

Another  critical  function  of  the  analyst  is  the  defini¬ 
tion,  for  the  decision  maker,  of  suitable  alternative 
systems  and  methods  of  operation.  The  analyst  should 
consider  a  large  enough  study  context  to  allow  for 
development  of  feasible  alternatives  and  trade-offs 
and  to  avoid  unsuitable  sub-optimal  decisions.  For 
instance,  different  time  phasings  of  any  one  program 
are  important  alternatives  to  be  considered.  However, 
ih:  cost  and  effectiveness  of  alternative  programs  are 
also  usually  very  relevant. 

The  analyst  should  call  the  decision  maker’s  atten¬ 
tion  to  uncertainties  in  his  study  and  should,  where 
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possible,  provide  him  with  sensitivity  analyses  for 
important  but  uncertain  phasing,  effectiveness  and  cost 
factors. 

Finally,  he  should,  where  possible,  indicate  which 
factors  cannot  be  analyzed  quantitatively  within  the 
time  frame  of  the  study  or  the  context  of  the  data 
available  and,  therefore,  should  be  left  to  the  judgment 
of  the  decision  maker. 

Background  jor  a  case  study 

Our  office  recently  had  the  opportunity  to  analyze 
some  interesting  information  systems  problems  dealing 
with  the  telecommunications  used  for  dispatch  and 
control  of  electrical  power  networks.  1  intend  to  use 
these  as  a  case  study  here  to  show  how  system  analysis 
is  being  generally  applied  to  the  acquisition  of  infor¬ 
mation  systems  throughout  the  Federal  Government.3 

In  the  case  under  consideration  a  program  decision 
had  to  be  made  whether  to  buy  a  microwave  system 
or  to  use  common  carrier  circuits  to  control  the  in- 
tertie  of  electrical  power  networks  in  two  widely 
separated  areas  of  the  country.  Both  Government- 
owned  and  privately-owned  systems  were  involved. 
The  purpose  of  these  interties  is  to  distribute  to  power- 
deficient  areas  the  extra  power  potential  in  rivers  that 
is  available  from  the  spring  runoff  of  melting  snows. 
A  number  of  electrical  power  utility  companies  arc 
participating  in  intcrtics  of  this  type.  Most  cases,  it 
turns  out,  involve  many  complex  technical,  legal  and 
economic  problems. 

It  is  well  known,  of  course,  that  public  utility  com¬ 
panies,  in  general,  differ  from  other  types  of  businesses. 
They  supply  transportation,  water,  gas,  electrical  pow¬ 
er,  telecommunications  and  other  types  of  services  that 
are  required  by  the  entire  community.  In  order  to  pro¬ 
vide  a  universally  high  quality  of  service,  to  avoid 
redundancy,  and  to  produce  economies  of  scale,  these 
companies  are  usually  legal  monopolies.  They  arc 
granted  franchises  to  operate  in  any  given  area.  In 
return,  the  Government  retains  the  right  to  regulate 
their  standards  of  service  and  their  tariff  rates. 

Normally,  each  type  of  public  utility  uses  services 
supplied  by  the  others.  For  instance,  the  telephone 
utilities'  primary  source  of  power  is  obtained  from  the 
electrical  power  utilities.  In  addition,  telephone  com¬ 
panies  maintain  standby  generators  and  batteries  to 
provide  backup  power  in  case  their  primary  sources  arc 
disrupted,  such  as  occurred  last  Fall  in  the  North¬ 
eastern  part  of  the  United  States. 

Many  of  the  power  utility  companies  obtain  their 
telecommunications  for  primary  control  of  their  power 
networks  from  the  telephone  companies  and  find  that 
this  type  of  service  satisfies  their  requirements.  How¬ 
ever,  many  other  power  utilities  throughout  the  United 


States  currently  own  and  operate  microwave  telecom¬ 
munication  systems  to  provide  primary  communications 
for  control  of  their  power  networks.  The  total  invest¬ 
ment  in  microwave  systems  in  the  United  States  for 
this  purpose  is  estimated  at  several  hundred  million 
dollars  and  can  be  expected  to  increase  as  large  elec¬ 
tronic  computers  arc  applied  in  the  future  to  control 
systems  for  these  types  of  networks.  Backup  telecom¬ 
munications  for  power  control  are  now  provided  both 
by  some  power  line  carrier  telephones  owned  and 
operated  by  these  power  utilities  and,  in  many  cases, 
by  telecommunication  services  obtained  from  the  tele¬ 
phone  companies. 

The  power  companies  that  do  own  and  operate  their 
own  microwave  systems  use  many  arguments  for  doing 
so.  They  state  that  the  cost  of  owning  such  a  system 
is  lower  than  obtaining  this  service  from  the  telephone 
companies;  that  the  telephone  companies  are  not  com¬ 
pletely  responsive  to  their  requirements,  particularly 
with  respect  to  the  reliability  and  availability  of  com¬ 
munications  that  they  need  for  vital  links  for  control 
of  their  power  system.  They  point  out  that  the  power 
networks  use  extremely  high  voltages  which  present 
many  safety  problems  and  are  difficult  to  control.  They 
are  particularly  concerned  with  the  safety  of  repair  and 
maintenance  men  working  on  transmission  lines,  the 
stability  of  the  AC  power  networks,  and  the  proper 
performance  of  the  new  types  of  DC  power  intertie 
transmission  lines.  They  stress  the  fact  that  they  require 
overall  system  responsibility  for  their  power  networks. 
The  normal  mode  of  operation  for  present  day  systems 
is  necessarily  automatic  and  requires  an  integrated 
communications  and  control  system  with  fast  reaction 
and  rapid  response  to  control  signals.  For  instance,  in 
systems  containing  about  1,000  miles  of  electrical 
power  transmission  lines,  surges  in  the  networks, 
caused  by  lightning  and  other  disturbances,  will  build 
up  to  a  maximum  in  about  half  of  a  second.  Therefore, 
circuit  breakers  must  be  operated  throughout  the  sys¬ 
tem  in  a  few  tenths  of  a  second,  to  prevent  these  surges 
from  burning  out  critical  components  such  as  trans¬ 
formers,  generators  and  insulators.  As  more  intcrtics 
are  created  throughout  the  United  States  and  Canada, 
the  intertied  networks  will  become  larger  and  more 
stable.  Simulation  models  indicate  that  the  time  con¬ 
stants  of  the  surges  will  increase.  Under  these  conditions 
the  overall  reaction  time  for  circuit  breakers  will  not 
need  to  be  as  fast  as  for  the  uncoordinated  networks. 
But  the  overall  control  of  the  larger  networks  will  be¬ 
come  more  important  since,  when  intertied,  larger 
sections  of  the  country  could  be  blacked  out  if  control 
of  the  intertied  network  is  disrupted. 

On  the  other  hand,  the  telephone  industry’s  position 
is  that  their  companies  are  public  utilities  too,  and  that 
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they  have  been  granted  franchises  to  supply  common 
carrier  services  throughout  specified  areas.  They  have 
large  pools  of  specially  trained,  professional  com¬ 
municators  for  maintenance  and  operation  of  existing 
systems  and  for  R&D,  design  and  engineering  of  new 
systems.  They  have  a  large  dispersed  network  of  com¬ 
munications  and  a  sizable  staff  of  maintenance  men 
and  extra  equipment  that  can  all  be  used  to  reroute 
and  restore  circuits  that  arc  disrupted  in  times  of  dis¬ 
aster.  They  argue  that  they  have  economies  of  scale 
that  produce  cost  advantages  over  smaller  privately- 
owned  or  Government-owned  systems.  Further,  they 
say  that  they  fully  realize  the  critical  nature  of  the 
power  utility  communications  and  in  recent  years  have 
taken  special  steps  to  improve  their  service  through, 
for  instance,  the  use  of  special  operational  control 
rooms  for  monitoring  and  managing  the  restoration  of 
important  telecommunication  circuits  for  electrical 
power  control. 

ODTM  systems  analysis 

In  cases  of  this  type  the  Office  of  the  Director  of 
Telecommunications  Management  (ODTM)  recognizes 
and  emphasizes  the  importance  of  preserving  the  Gov¬ 
ernment’s  freedom  of  choice  between  obtaining  common 
carrier  services  or  buying  Government-owned  tele¬ 
communications  systems.  This,  it  is  felt,  makes  it  pos¬ 
sible  for  the  Government  to  take  into  account  all 
current  factors  in  making  a  decision  on  how  to  obtain 
satisfactory  services  for  the  lowest  cost.  However,  as  a 
general  rule,  the  policy  of  the  Federal  Government,  as 
stated  in  Bureau  of  the  Budget  Circular  A -76,  is  that 
the  Government  should  rely  on  the  private  enterprise 
system  to  supply  its  needs. 

Exceptions  to  this  policy  cited  in  the  Circular  are 
cases  where: 

•  Procurement  of  a  product  or  service  from  a 
commercial  source  would  disrupt  or  materially 
delay  the  agency’s  program. 

•  It  is  necessary  for  the  Government  to  conduct 
a  commercial  or  industrial  activity  for  purposes 
of  combat  support  or  for  individual  and  unit 
retraining  of  military  personnel  or  to  maintain 
or  strengthen  mobilization  readiness. 

•  A  satisfactory  commercial  source  is  not  avail¬ 
able  and  cannot  be  developed  in  time  to  pro¬ 
vide  the  product  or  service  when  it  is  needed. 

•  The  product  or  service  is  available  from  an¬ 
other  Federal  Agency. 

•  Procurement  of  the  product  or  service  from 
a  commercial  source  will  result  in  a  higher 
cost  to  the  Government. 

Interpreting  the  applicable  policy  in  terms  of  the 
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case  studied,  indicates  that  the  Government  should 
lease  the  intertie  microwave  system  unless  one  or  more 
of  the  following  conditions  exist: 

•  The  telephone  companies  that  proposed  to 
supply  the  service  could  not  meet  the  schedule 
phasing  deadline  for  completion  of  the  electric 
power  intertie. 

•  The  telephone  companies  could  not  supply  an 
effective  telecommunications  system  for  con¬ 
trol  of  the  electric  power  system. 

•  The  telephone  companies’  service  would  cost 
more  than  a  Government-owned  telecommuni¬ 
cations  system. 

The  applicability  of  these  conditions  formed  the 
basis  for  our  systems  analysis  of  these  cases. 

Schedule  phasing 

Analysis  of  the  first  condition  showed  that  although 
the  initial  investment  in  the  telecommunications  for 
the  electrical  power  intertie  would  cost  only  about  $2 
million,  the  estimated  loss  in  revenues  would  amount 
to  almost  $0.5  million  per  month  if  adequate  control 
mechanisms,  including  required  telecommunications, 
were  not  completed  by  the  time  the  power  intertie  was 
ready  for  operation.  Schedule  phasing  for  telecommuni¬ 
cations  was,  therefore,  a  critical  although  not  an  over¬ 
riding  factor  in  the  decision. 

Because  the  telephone  companies  planned  to  use 
existing  facilities  for  part  of  the  required  system,  and 
because  of  preplanning  and  scheduling  flexibility  in 
the  growth  of  their  nationwide  microwave  system,  the 
telephone  companies  were  willing  to  commit  them¬ 
selves  to  meeting  the  operational  deadlines  for  the 
power  intertie  at  the  time  the  decision  was  made  to 
proceed  with  the  acquisition  of  the  telecommunications 
system.  The  suppliers  of  equipment  for  the  proposed 
Government-owned  systems,  on  the  other  hand,  could 
not  guarantee  an  operational  telecommunications  system 
until  about  45  days  after  this  deadline,  although  they 
felt  they  might  be  able  to  decrease  their  time  require¬ 
ments  if  production  scheduling,  and  possible  installation 
delays  caused  by  weather,  permitted. 

Effectiveness  of  telecommunications 

As  indicated  previously,  one  of  the  most  critical 
functions  of  telecommunications  in  this  type  of  task 
is  to  provide  a  means  for  disconnecting  an  AC  power 
network  if  transients  or  surges  threaten  to  overload  and 
burn  out  the  elements  of  the  network.  A  simplified 
diagram  of  a  typical  control  system  is  shown  in  Figure 
diagram  of  a  typical  control  system  is  shown  in  Figure  1. 

If  a  transient  occurs,  an  overload  detector,  such 
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Location  No.  1  Location  No.  2 

▼  ▼ 


Figure  1 — Typical  power  line  control  system 


as  a  potential  or  current  transformer,  generates  a  sig¬ 
nal  that  is  fed  to  a  tone  equipment  where  it  is  used  to 
modulate  a  carrier  frequency.  The  output  of  the  tone 
equipment  is  transmitted  over  a  communications  sys¬ 
tem  to  a  tone  equipment  at  another  location.  Here  the 
earrier  is  demodulated  and  the  signal  used  to  actuate  a 
circuit  breaker  for  disconnecting  power  line  loads  or 
dropping  power  generators  at  that  point. 

The  objectives  of  the  telecommunications  portion  of 
this  system  are,  therefore,  to  insure  that  elements  of  the 
electrical  power  system  will  not  be  destroyed  by  in¬ 
sufficient  rapidity  of  control  response  and  that  tele¬ 
communications  reliability  will  not  significantly  degrade 
the  reliability  of  the  system  to  either  act  when  action 
is  required  or  to  not  act  when  action  is  not  required. 

Measures  of  effectiveness  of  the  system  that  were 
studied,  therefore,  included: 

•  Time  delays  from  detection  of  an  overload  in 
power  transmission  line  at  Location  No.  1  to 
the  completion  of  the  circuit  breaker  action  at 
Location  No.  2. 

•  Reliability  of  the  system  in  actuating  the  cireuit 
breaker  when  an  overload  signal  is  received. 

•  The  probability  that  noise  generated  in  the 
system  will  not  accidentally  trip  the  circuit 
breaker. 

It  is  important  to  investigate  not  only  the  effective¬ 
ness  and  uncertainties  of  the  telecommunications  por¬ 
tion  of  the  system  but  also  the  overall  effectiveness  and 
uncertainties  of  the  system  described  in  Figure  1. 

The  relative  contribution  to  the  overall  time  delay 
of  the  various  telecommunication  system  alternatives 
under  consideration  can  then  be  determined.  Also  it  is 
important  to  compare  the  uncertainties  in  effectiveness 
of  the  overall  system  with  the  variations  in  effectiveness 
produced  by  the  alternative  modes  of  telecommunica¬ 
tions  in  order  to  determine  the  relative  benefits  of  in¬ 
troducing  eaeh  of  these  alternatives  into  the  system. 
Time  delays 


The  time  delay  in  the  system,  Te,  will  be 

m 

t.=2t- 

t 

where  T„  =  the  time  delay  in  the  nth  component  in 
system. 

m  =  the  number  of  components  in  the  sys¬ 
tem. 

In  other  words,  the  total  time  delay  is  equal  to  the 
sum  of  the  time  delays  in  the  individual  components. 
Typical  time  delays  are  shown  in  Figure  2  for  the 
components  of  the  system  indicated  in  Figure  1. 

Our  analysis  indicated,  therefore,  that  the  time  delays 
in  the  overload  deteetor  and  the  cireuit  breaker  and 
the  uncertainties  in  these  time  delays  overshadowed 
the  time  delays  and  uncertainties  in  the  communications 
portion  of  the  system  (i.e.,  the  two  tone  equipments  and 
the  microwave  link).  An  obvious  conclusion,  therefore, 
is  that,  if  reductions  in  total  time  delays  and  time 
delay  uncertainties  are  required,  improvements  should 
be  made  in  the  overload  detectors  and  circuit  breakers 
rather  than  in  the  telecommunications  portion  of  the 
system.  For  instance,  one  issue  that  arose  during  the 
analysis  was  whether  wideband  tone  equipments  should 
be  used  to  reduce  total  time  delays  in  these  components 
from  between  22  and  24  ms  to  between  14  and  16  ms. 
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Figure  2 — Time  delays  in  system  components 


Our  analysis  indicated  that  while  this  improvement 
would  be  desirable,  high  quality  voice  band  communi¬ 
cation  links  and  terminals  would  not  improve  the 
quality  of  the  circuit  by  any  significant  amount.  This 
point  is  discussed  in  more  detail  in  a  seetion  entitled 
44  Noise/* 

Another  issue  that  was  related  to  the  decision  was 
whether  microwave  would  be  required  throughout  the 
network,  and  particularly  on  the  longer  links,  or  wheth¬ 
er  toll  eable  links  eould  be  used  without  degrading  the 
system  performance.  Typical  time  delays  for  microwave 
links  are  about  2  rm  per  hundred  miles  of  route  and 
for  loaded  toll  cable  systems  about  6  ms  per  hundred 
miles  of  route.  It  was  estimated,  therefore,  that  for 
control  circuits  not  greater  than  about  300  miles  in 
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length,  either  loaded  toll  cables  or  microwave  or  a 
mixture  of  the  two  could  be  used  interchangeably  with¬ 
out  significantly  degrading  the  time  delay  of  the  total 
system.  Only  in  exceptional  cases  will  these  types  of 
control  circuits  be  greater  than  300  miles  in  length. 

In  the  case  studied,  both  the  Government-owned  and 
the  common  carrier  alternatives  proposed  to  use  micro- 
wave  only.  Some  additional  delays  would  have  to  be 
added  to  the  common  carrier  circuits  because  some  of 
the  links  would  pass  through  telephone  company  central 
offices.  However,  the  common  carrier  network  alter¬ 
native  was  designed  to  minimize  these  types  of  delays. 
In  no  case  did  the  planned  total  time  delay  exceed  5ms 
for  control  circuits  in  this  alternative.  This  factor,  there¬ 
fore,  was  not  considered  to  be  significant  in  reaching  a 
decision  to  buy  or  lease  the  system. 

Reliability 

The  composite  reliability  R0  of  the  system  shown  in 
Figure  1  will  be 

m 


1 


where  Rn  ==  the  reliability  of  the  nth  component  of 
the  system. 

In  other  words,  the  composite  reliability  of  the  system 
will  be  the  product  of  the  reliabilities  of  the  individual 
components.  Typical  reliabilities  of  components  are 
shown  in  Figure  3  for  the  system  indicated  in  Figure  1. 
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Figure  3 — Reliability  of  system  components 

A  study  has  been  made  that  compared  the  reliability 
of  38  Government-owned  telecommunication  circuits 
with  46  common  carrier  circuits  over  the  period  of  one 
year.  Each  common  carrier  circuit  passed  through  an 
average  of  three  telephone  company  central  offices.  The 
Government-owned  circuits  did  not  pass  through  any 
central  offices.  Figure  4  compares  the  number  of  out¬ 
ages  per  year  and  the  outage  durations  for  these  two 
categories  of  circuits.  Although  the  average  rate  of 
outages  was  about  the  same  for  the  two  types,  the 
average  outage  duration  of  the  common  carrier  circuits 
is  seen  to  be  significantly  smaller  than  the  Government- 


owned  circuits.  In  other  words,  whenever  an  outrage 
occurred  it  took  less  time  for  the  telephone  companies 
to  fix  their  circuits  than  for  the  Government  to  fix  its 
circuits.  The  average  reliability  of  the  Government- 
owned  circuits  was  found  to  be  0.9963  compared  to 
0.9997  for  common  carrier  circuits. 

Dual  routing,  using  both  microwave  and  cable,  can 
be  used  to  improve  the  reliability  of  any  telecommuni¬ 
cations  system,  if  required.  This  has  been  successfully 
accomplished  by  the  Power  Authority  of  the  State  of 
New  York  and  others.  The  reliability  of  dual  routed 
circuits,  assuming  that  they  are  completely  independent, 
can  be  expressed  as  follows: 

R,  =  1  -(1  -  Rv)  (1  -R.) 

where  Ry  and  R*  are  the  reliabilities  of  each  of  the 
dual  circuits.  For  instance,  if  Ry  =  0.9963  and  R«  = 
0.9997,  then  R*  =  0.999999.  Another  way  of  looking 
at  this  is  that  it  would  reduce  the  expected  outage  time 
from  1,945  minutes  per  year  and  158  minutes  per 
year  on  the  individual  circuits,  to  only  about  1  minute 
per  year  on  the  dual  routed  circuit. 

However,  the  important  conclusion  here  is  that  no 
significant  gain  in  overall  system  reliability  can  be 
achieved  by  increasing  the  reliability  of  the  communi¬ 
cations  components  alone,  if  the  reliability  of  the 
overload  detector,  the  circuit  breaker  and  other  system 
components  cannot  be  increased  to  a  comparable  level. 
For  instance,  in  the  case  studied,  if  the  communications 
system  reliability  is  increased  to,  say  0.9999  the  re¬ 
liability  of  the  overall  system  would  still  be  less  than 
0.98. 

Noise 

The  composite  probability,  PQ,  that  the  circuit  break¬ 
er  will  not  be  accidentally  tripped  by  noise  in  any  of  the 
elements  of  the  system  is 

m 

-IP- 

1 

where  Pn  =  the  probability  that  noise  generated  in 
the  nth  component  of  the  system  will  not  trip  the  circuit 
breaker. 

The  tests  described  below  were  used  to  determine 
conditions  under  which  noise,  introduced  into  the 
system  by  the  telecommunications  components,  could 
trip  the  circuit  breaker.  Unfortunately,  no  data  were 
available  to  determine  the  probability  of  tripping  the 
circuit  breaker  by  noise  generated  in  the  non-telecom¬ 
munications  components  of  the  system. 

Actually,  it  was  planned  to  use  the  same  type  of 
tone  equipment  in  either  the  Government-owned  or 
the  common  carrier  system.  The  main  issue,  therefore, 
was  whether  conventional  telephone  channels  could  be 
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Figure  4 — Comparison  of  Outrages  for  Government- 
owned  and  Common  Carrier  Circuits 


used  or  whether  high-quality,  delay-equalized  telecom¬ 
munications  links  would  be  required. 

The  tests  were  made  over  a  350-mile  circuit.  The 
tone  equipment  was  lined  up  according  to  the  man¬ 
ufacturers’  specifications.  Tests  were  performed  to 
determine  squelch  activation  time  and  squelch  recovery 
time  for  both  impulse  and  white  noise,  the  effect  of 
noise  on  false  trip  operation  of  the  eircuit  breaker  as 
well  as  range  of  reliable  trip  operation.4 

The  operation  of  the  system  was  compared  using  the 
conventional  telephone  channel  and  the  high-quality, 
delay-equalized  channel.  The  false  trip  levels  were  ob¬ 
tained  by  deactivating  the  squelch  unit  in  the  tone 
equipment  on  the  receiving  end  and  introducing  im¬ 


pulse  or  flat  noise  at  various  points  in  the  system. 
This  noise  was  introduced  at  both  the  output  of  the 
transmitting  tone  equipment  and  the  input  of  the  re¬ 
ceiving  tone  equipment. 

It  was  found  that  the  tone  equipment  was  eapable 
of  operating  with  impulse  noise  levels  in  excess  of 
signal  levels  before  false  operation  occurred.  Impulse 
noise  eould  exceed  white  noise  by  several  db  before 
false  trips  occurred.  A  squelch  setting  level  of  at  least 
7  db  below  the  normal  value,  for  false  operation  from 
cither  white  or  impulse  noise  was  used  as  a  margin  of 
safety  to  allow  for  squelch  threshold  drift. 

With  the  system  maintained  at  the  normal  levels  a 
period  of  operation  of  four  months  resulted  in  only 
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one  false  operation.  This  was  caused  by  a  frequency 
shift  in  the  tone  equipment  resulting  from  partial  power 
failure  rather  than  from  the  effects  of  any  noise  in  the 
system. 

Our  general  conclusions  from  these  tests  were  that 
the  common  carrier  service  and  the  Government-owned 
system  would  be  equally  effective  in  rejecting  noise  that 
might  cause  false  operation  of  the  circuit  breaker  and 
that  conventional  telephone  circuits  would  be  adequate 
for  the  degree  of  reliability  required  by  the  system. 

Cast  comparison 

The  final  question  posed  by  Circular  A-76  was 
whether  the  telephone  companies’  service  would  cost 
more  than  a  Government-owned  telecommunications 
system. 

The  method  of  estimating  annual  costs  for  the 
Government-owned  system  as  outlined  in  Circular 
A-76  is  first  to  determine  the  composite  life  of  the 
system,  including  buildings,  access  roads,  antennas, 
transmission  facilities,  etc.  and  then  to  amortize  the 
initial  investment  over  the  period  of  this  composite 
life.  In  determining  the  amortization  cost,  the  interest 
rate  must  be  for  the  system  must  be  discounted  to  its 
value  at  the  equivalent  to  “the  current  rate  for  long-term 
Treasury  obligations  for  capital  items  having  a  useful 
life  of  15  years  or  more  and  upon  the  average  rate  of 
return  on  Treasury  obligations  for  items  having  a  useful 
life  of  less  than  15  years.”  Circular  A-76  also  stipulates 
that  allowances  should  be  included  for  annual  opera¬ 
tions  and  maintenance  costs  and  Federal  taxes  foregone. 
In  addition  it  requires  that  “new  starts  ordinarily  should 
not  be  approved  unless  cost  of  a  Government  activity 
will  be  at  least  10  per  cent  less  than  costs  of  obtaining 
the  product  service  from  commercial  sources.” 

The  cost  of  the  common  carrier  service  on  the  other 
hand  should  be  based  on  the  existing  tariff  rates  for 
the  service  provided,  together  with  any  additional  costs 
not  covered  by  the  tariffs,  such  as  engineering  costs  and 
Government  administration  of  the  service  contract.5 

The  principal  cost  issues  that  arose  in  the  Govern¬ 
ment-owned  case  concerned  the  determination  of  an 
appropriate  value  for  the  composite  life  of  the  system 
and  the  estimation  of  the  annual  cost  of  operations  and 
maintenance  for  the  system. 

The  arguments  for  Government  ownership  centered 
around  the  fact  that  a  transistorized  telecommunications 
system  was  specified  that  would  have  longer  life  and 
lower  operations  and  maintenance  costs  than  the  sys¬ 
tems  previously  used. 

On  the  other  hand,  the  telephone  companies  argued 
that  their  quotation  must  be  based  on  their  tariff  rate 
structure  which  must  necessarily  reflect  the  cost  of  all 
of  their  facilities.  These  included  some  transistorized 


equipments  but  their  system  is  still  mainly  non-trans- 
istorized.  Therefore,  their  quotation  did  not  reflect 
possible  future  reductions  in  tariffs  resulting  from  these 
types  of  improvements. 

The  position  of  our  office  was  that  since  both  the 
Government-owned  and  common  carrier  systems  would 
employ  transistorized  equipment  the  comparisons 
should  be  made  either  on  the  basis  of  present  overall- 
experience  on  composite  system  life  and  operation 
and  maintenance  for  both  cases  or  that  both  should 
anticipate  decreased  costs  in  the  future  because  of  the 
shift  to  improve  transistorized  systems.  On  these  bases 
the  annual  cost  of  the  two  systems  was  estimated  by 
our  office  to  be  about  equal  in  the  two  cases,  excluding 
the  10  per  cent  margin  factor  stipulated  by  Circular 
A-76. 

In  an  attempt  to  develop  a  better  cost  methodology 
for  these  types  of  decisions  our  office  has  contracted 
with  the  Planning  Research  Corporation.  They  arc 
developing  alternative  means  of  making  lease-versus- 
buy  cost  comparisons.  One  of  the  methods  being  studied 
is  to  discount  the  stream  of  cost  differences  throughout 
the  life  of  alternative  systems  and  then  derive  a  “rate 
of  return  on  investment”  as  a  means  of  comparing  the 
worth  of  the  systems. 

Other  issues 

The  above  discussion  includes  only  a  few  of  the 
more  important  issues  that  were  considered  in  this 
case.  There  were,  in  addition,  many  dealing  with  legal 
factors  as  well  as  the  question  of  the  responsiveness 
of  the  various  bids  submitted. 

For  instance,  this  case  illuminated  a  number  of 
tariff  problems,  including  the  wide  disparity  between 
intrastate  and  interstate  TELPAK  service  rates.  Fur¬ 
thermore,  it  was  found  that  differences  in  composite 
depreciation  lives  resulting  from  different  mixes  of 
equipment  are  not  reflected  in  the  current  interstate  and 
intrastate  tariffs. 

Another  issue  that  arose  during  the  analysis  con¬ 
cerned  the  legality  of  leasing  circuits  in  a  Government- 
owned  microwave  system  to  privately-owned  public 
utilities  rather  than  requiring  the  utilities  to  lease  these 
circuits  from  the  common  carriers. 

At  the  present  time  some  confusion  seems  to  exist 
concerning  the  accounting  of  Federal  taxes  in  these 
cost  comparisons.  In  accordance  with  the  guidance  sup¬ 
plied  in  BoB  Circular  A-76,  Federal  taxes  foregone 
should  normally  be  added  to  the  cost  of  a  Government- 
owned  system,  if  the  comparable  common  carrier 
service  would  operate  with  a  profit.  An  issue  raised 
in  the  analysis  was — if  TELPAK  offerings  do  not  pro¬ 
duce  a  profit  for  the  telephone  company  should  these 
Federal  taxes  be  included  in  the  cost  of  the  Govern- 
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ment-owned  system?  The  telephone  industry  argued 
that  the  Federal  tax  applies  only  to  the  gross  revenue 
of  the  company.  On  the  other  hand  the  electrical  power 
utilities  argued  that  the  telephone  industry  is  not  making 
a  profit  on  TELPAK  and,  therefore,  that  Federal  taxes 
foregone  should  not  be  included  in  the  cost  of  the 
Government-owned  system. 

Another  important  issue  arose  in  regard  to  system 
design  and  engineering  costs.  Costs  incurred  by  the 
Government  for  these  types  of  services  have  been  ap¬ 
plied  by  the  electrical  power  utility  to  the  cost  of  the 
common  carrier  service.  The  telephone  industry  claims 
that  system  design  and  engineering  services  are  normally 
performed  by  the  telephone  industry  and  that  costs  for 
these  services  are  covered  by  the  tariffs. 

With  respect  to  the  responsiveness  of  the  telephone 
industry  bid  for  the  intertie  telecommunications,  our 
study  concluded  from  the  evidence  gathered,  that  the 
bid  invitation  was  technically  ambiguous  in  several  ways. 
Many  of  the  principal  legal  issues  that  were  raised  in 
this  case  have  also  arisen  previously  in  similar  Federal 
Government  contracts  with  the  telephone  industry. 
They  were  resolved  to  the  satisfaction  of  the  Govern¬ 
ment  and  the  telephone  industry. 

It  would  certainly  be  helpful,  in  the  future,  if  specific 
policies  could  be  established  which  would  clarify  the 
types  of  exceptions  that  make  a  proposal  unresponsive 
to  a  telecommunications  bid  invitation  and  to  outline 
in  more  detail  the  procedures  that  should  be  followed 
when  a  decision  of  non-responsiveness  results  from  am¬ 
biguities  in  the  invitation  to  bid. 

It  appears,  further,  that  standards  are  needed  for  the 
design  of  bid  invitations  for  common  carrier  services. 
It  should  be  recognized  that  specifications  for  special¬ 
ized  telecommunications  services,  such  as  those  de¬ 
signed  to  meet  the  needs  for  electrical  power  interties, 
may  require  careful  negotiation  between  the  Govern¬ 
ment  and  the  telephone  industry  before  the  bid  invita¬ 
tion  is  issued. 

In  looking  at  the  long-range  issues  associated  with 
power  utility  telecommunications  there  is  one  that 
appears  to  be  of  particular  significance.  In  its  bid 
invitation  the  electrical  power  utility  specified  that  mi¬ 
crowave  circuits  must  be  used  throughout  the  system  to 
reduce  transmission  time  delays.  However,  in  the  future, 


spectrum  crowding  may  force  the  privately-owned  and 
Government-owned  systems  as  well  as  the  telephone 
industry  to  convert  to  telecommunications  cable  sys¬ 
tems.  There  is  an  indication  that  the  power  control 
systems  may  in  some  long  distance  circuits,  require 
faster  reaction  times  than  could  be  supplied  by  cable 
systems.  However,  as  indicated  previously  the  exten¬ 
sion  and  intertying  of  the  power  grids  will  increase  the 
critical  time  constants  of  the  system.  The  tradeoffs  of 
these  factors  should  be  the  subject  of  continuing  study 
as  the  network  configuration  and  the  spectrum-crowding 
situations  change. 

Because  of  the  close  interdependence  of  the  telephone 
industry  and  the  power  industry,  as  has  been  noted 
above,  consideration  should  be  given  to  the  formation 
of  an  analytical  and  policy  making  task  force  that  can 
be  asked  to  determine  the  most  effective  means  of 
assigning  national  assets,  such  as  men,  materials  and 
spectrum  space  to  the  nation’s  composite  telecom¬ 
munications  system  including  those  of  both  the  tele¬ 
phone  and  the  power  utilities. 

Our  office  plans  to  continue  its  study  of  these  types 
of  cases  since  we  feel  that  the  lessons  learned  will 
prove  useful  in  formulating  adequate  policies  for  these 
types  of  telecommunications  issues.  In  addition,  an 
Intra-Governmental  Study  Group,  with  representatives 
from  12  Federal  Departments  and  Agencies,  has  been 
established  to  assist  in  developing  and  coordinating 
further  guidance  for  these  types  of  decisions  as  they 
pertain  to  Federal  telecommunications  systems. 
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The  future  effectiveness  of  multiple-access  com¬ 
puter  systems  as  aids  to  the  individual  will  greatly 
depend  on  how  well  we  will  be  able  to  match  them  to 
their  users,  to  their  individual  needs,  and  to  their 
community  goals.  There  are  many  aspects  to  this 
matching  problem.  The  first  and  most  elementary 
one  is  the  instructional  aspect.  Since  the  main  value 
of  a  computer  system  lies  in  the  great  wealth  and 
variety  of  services  that  it  can  provide,  learning  how 
to  exploit  them  will  be  a  continuing  process  for  all 
users.  In  particular  a  user  should  be  able  to  learn,  and 
receive  help  when  in  trouble,  from  a  remote  loca¬ 
tion.  without  depending  on  any  next-door  expert. 
The  second  aspect,  somewhat  more  subtle,  is  the  trust 
that  each  user  can  be  expected  to  develop  toward 
the  computer  system:  trust  that  it  will  be  available 
when  needed,  trust  that  it  will  not  lose  the  products 
of  his  work,  trust  that  it  will  protect  his  work  and  his 
private  affairs  against  malicious  or  negligent  actions 
on  the  part  of  other  users,  and  trust  that  other  users 
are  not  given  economic  advantages  or  other  privileges 
at  his  expense.  The  ramifications  of  these  issues  go 
far  into  the  design  of  computer  systems  and  into  their 
operating  policies  and  procedures.  The  final  and  most 


crucial  aspect  concerns  the  reciprocal  influence  of 
the  intellectual  complexion  of  the  community  of  users 
and  the  nature  of  the  facilities  offered  by  the  computer 
system  serving  them.  There  are  indications  that  the 
coupling  between  system  and  community  may  be  so 
strong  as  to  quickly  magnify  initial  characteristics  of 
either  of  them.  For  instance,  if  the  system  happens 
to  be  especially  effective  for  some  particular  type  of 
work,  that  type  of  work  will  prosper  and  even  better 
facilities  for  it  will  become  available.  Conversely, 
activities  for  which  the  system  is  initially  poorly 
suited  will  be  discouraged  and  may  eventually  die 
out.  This  phenomenon  of  selective  reinforcement 
may  well  change  in  a  radical  manner  the  intellectual 
profile  and  character  of  a  community. 

The  importance  of  properly  matching  a  computer 
system  to  its  community  of  users  cannot  be  over¬ 
emphasized.  As  a  matter  of  fact,  we  may  well  have 
to  think  in  the  future  of  the  community  of  users  as 
part  of  the  system  itself,  or,  perhaps  more  appropri¬ 
ately,  of  the  system  as  the  repository  of  the  com¬ 
munity’s  knowledge,  and  as  the  major  channel  for 
intellectual  communication  between  its  members. 
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INTRODUCTION 

This  paper  proposes  an  experiment  in  communication 
in  the  computer  science  field.  The  experiment  consists 
in  the  development  of  a  number  of  vehicles  for  inter¬ 
changing  information  amongst  a  group  of  computer 
users  and  developers.  Each  vehicle  is  designed  to  satis¬ 
fy  a  particular  need  and  the  new  features  of  the  experi¬ 
ment  lie  in  the  putting  together  of  a  number  of  vehicles 
rather  than  in  the  novelty  of  any  particular  vehicle. 
The  choice  of  computer  science  as  a  technical  area  in 
which  to  experiment  was  made  for  several  reasons. 
There  is  a  clear  and  great  need  for  better  communica¬ 
tion  techniques  in  this  field.  A  number  of  communica¬ 
tion  vehicles  involve  the  use  of  computers  and  com¬ 
puter  files  so  the  technology  to  develop  these  vehicles 
is  available  in  the  computer  users  group.  However, 
although  the  vehicles  described  here  will  be  specific 
to  computer  science,  many  of  the  ideas  involved  could 
be  generalized  to  other  technical  areas. 

The  particular  characteristics  of  computer  science 
which  will  be  used  as  a  basis  for  proposing  communica¬ 
tion  vehicles  are: 

•  It  is  possible  to  delineate  fairly  sharply  the  limits 
of  the  subjects  in  the  field, 

•  The  field  is  changing  very  rapidly,  so  that  speed 
in  communications  is  essential. 

•  There  is  a  real  need  to  disseminate  information 
amongst  a  group  of  very  interested  workers. 

In  order  to  make  the  vehicle  seem  somewhat  more 
realistic,  I  will  take  as  a  specific  environment  the 
development  of  the  second  generation  time  shared 
computer  with  remote  access.  This  work  has  produced 
a  great  deal  of  discussion,  soul-searching,  confusion, 
and  general  exchange  of  information  and  thoughts 
amongst  the  communities  which  are  involved.  Inherent 
in  the  process  arc: 

•  The  development  of  the  computer  itself.  Sub¬ 
stantial  changes  have  been  made  in  the  original 
computing  machines  as  a  result  of  deliberations. 


•  Writing  the  system  programs  which  will  make 
the  computing  machine  available  to  a  wide  group 
of  users,  particularly  by  means  of  remote  con¬ 
soles  using  time  sharing. 

•  Describing  these  programs  and  disseminating  in¬ 
formation  about  them  to  the  user  community. 

•  Incorporating  user  generated  needs  in  the  com¬ 
puting  system. 

•  Training  users. 

All  these  communication  requirements  are  engen¬ 
dered  by  the  new  machine  and,  furthermore,  they  are 
typical  of  any  large  transition  in  computer  systems. 

At  present  these  communication  requirements  are 
being  “satisfied"  in  a  haphazard  and  unsatisfactory 
way.  Decisions  are  made  by  various  committees,  but 
the  total  community  is  not  kept  apprised  of  the  delib¬ 
erations  or  made  sufficiently  aware  of  the  crucial 
issues.  Feedback  is  insufficient  from  the  general  com¬ 
munity  of  users  to  be  effective  in  directing  the  plan¬ 
ning  of  the  system.  After  the  planning  stage,  it  is 
necessary  to  disseminate  information  about  various 
programs  that  are  being  written.  Initially,  information 
must  go  to  people  writing  other  parts  of  the  system 
which  interact  with  these  programs;  eventually,  infor¬ 
mation  must  go  to  the  general  users  of  the  system. 
At  present,  the  adequate  transmission  of  this  informa¬ 
tion  is  too  dependent  on  the  writing  abilities  and  the 
time  taken  by  the  individual  programmers.  Many 
difficulties  and  mistakes  arise  from  this  poor  commu¬ 
nication;  these  range  from  errors  in  the  system  pro¬ 
grams  themselves  to  less  effective  use  of  the  system. 

The  communication  problems  in  a  computer  com¬ 
munity  are  made  more  difficult  by  the  rapidity  with 
which  programs  in  computers  change.  However,  they 
arc  typical  of  communication  problems  in  many  other 
areas  of  science  and  technology.  And  they  will  become 
more  typical  if  the  pace  of  technological  change  con¬ 
tinues  to  increase  and  the  number  of  people  engaged 
in  a  particular  technology  increases. 
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In  order  to  facilitate  this  communication  I  here 
pose,  as  examples,  seven  vehicles,  each  designed  to 
handle  one  particular  facet  of  the  problem.  The  vehi¬ 
cles  are  a  newspaper;  a  computer  user's  handbook;  a  set 
of  instructional  programs;  a  person  who  is  a  combina¬ 
tion  of  reference;  librarian,  instructor,  and  adviser;  a 
set  of  computer  programs;  definitive  summaries  of  the 
state  of  the  computation  art,  and  periodic  meeting  of 
a  computer  user's  society  where  oral  papers  are  given. 
The  rest  of  this  paper  will  outline  these  vehicles  in 
more  detail. 

Vehicle  1 — newspaper 

A  technical  newspaper  is  an  essential  medium  for 
rapid  dissemination  information.  Ordinary  newspapers 
ean  serve  as  a  model  for  this  deviee.  However,  hope¬ 
fully,  this  newspaper  will  be  more  accurate  than  ordi¬ 
nary  papers  sinee  it  will  be  directed  to  a  smaller  com¬ 
munity.  The  paper  will  be  published  for  news  value. 
Material  will  be  published  very  rapidly,  and  the  paper 
itself  will  not  be  intended  as  a  permanent  record. 
Thus,  every  effort  will  be  made  neither  to  save  nor  to 
index  nor  to  reference  nor  to  cite  material.  It  is  not 
always  possible  to  prevent  things  from  being  saved, 
particularly  in  a  library  environment.  However,  if  the 
functions  of  indexing,  referencing  and  citing  are  not 
done,  then  a  few  piles  of  newspaper  or  a  few'  rolls  of 
microfilm  will  not  cause  too  much  damage.  Indeed, 
these  may  be  of  value  as  ultimate  souree  material  to 
future  historians. 

As  an  experimental  comparison,  the  newspaper 
should  be  published  in  two  ways.  It  may  be  presented 
on  a  daily  or  weekly  computer  file,  and  distributed 
over  the  network  of  computer  consoles.  It  may  be 
published  in  a  paper  form  as  newspapers  are  pres¬ 
ently  put  out.  A  very  interesting  comparison  of  these 
two  media  for  very  rapid  dissemination  of  informa¬ 
tion  is  thus  possible. 

Four  types  of  material  will  typically  be  in  the  paper; 
news  items,  letters  to  the  editor,  articles,  and  adver¬ 
tisements.  I  will  give  some  indicative  examples  of  each: 

News  items — Typical  headlines  of  news  items  might 
be  as  follows:  Industry  and  schools  slug  it  out  over 
foreground  versus  background  computing;  computer 
manufacturer  accedes  to  user  demands  for  hardware 
page  tuner — a  new  computer  is  born!  Mrs.  System- 
Input-Output  pregnant  again — how  can  we  replace  our 
irreplaceable  programmers?  Intrex  conference  debates 
— should  we  go  time-shaped  or  have  a  separate  com¬ 
puter  facility  for  Intrex? 

Letters  to  editor — Typical  letters  might  eoneern  the 
following  subjects;  A  repartee  on  PL/1  versue  Comit 
2  for  text-editing;  A  letter  leading  for  adequate  com¬ 


putational  facilities  for  computer  music  on  the  new 
system. 

Articles — A  typical  article  by  Art  Telligenee  might 
start:  “I  just  had  a  great  idea.  Let's  write  the  new 
system  in  Lisp  3  because  ...” 

Advertisements  Many  computer  manufacturers 
would  find  a  newspaper  a  good  medium  for  advertis¬ 
ing.  These  advertisements  might  serve  a  good  purpose 
in  keeping  the  user  community  up  to  date  regarding 
equipment. 

Vehicle  2 — the  system  user's  handbook 

This  handbook,  which  would  be  in  the  form  of  a 
computer  file,  would  give  detailed  descriptions  of  the 
current  public  programs  in  the  system  and  the  current 
equipment  availaole  to  the  publie.  At  present  it  is 
difficult  to  maintain  up-to-date  information  about 
these  programs,  to  disseminate  this  information  and 
to  eliminate  previous  information  whieh  has  been  dis¬ 
seminated.  T  he  handbook  file  will  try  to  achieve  sever¬ 
al  objectives: 

•  It  will  be  kept  absolutely  current.  When  a  “pub- 
lie  program”  is  changed  the  description  in  the 
handbook  must  be  changed  at  the  same  time.  No 
introduction  of  new  public  programs  will  be  al¬ 
lowed  until  the  handbook  changes  are  completed 
for  concurrent  introduction. 

•  No  official  printed  version  of  the  handbook  will 
exist.  Thus  changing  the  one  computer  file  will 
be  enough  to  officially  erase  all  past  versions  of 
a  particular  set  of  information.  Of  eourse,  it  is 
impossible  to  prevent  some  people  from  making 
printouts  of  the  official  file.  But  they  do  so  at 
their  own  risk  and,  in  general,  such  printed  ver¬ 
sions  will  be  discouraged. 

•  The  user's  handbook  will  be  written  and  edited 
for  comprehensibility  to  the  “experienced”  sys¬ 
tem  user.  By  necessity,  a  section  describing  a  new 
program  will  be  written  by  the  programmer  pre¬ 
paring  the  program.  However,  to  achieve  the  re¬ 
quired  comprehensibility,  at  least  one  stage  of 
reading  and  revision  by  an  experienced  and  criti¬ 
cal  outside  programmer  or  editor  will  be  re¬ 
quired.  If  necessary,  several  stages  will  be  done. 

The  user's  handbook  will  be  indexed  in  depth  using 
all  available  and  useful  indexing  techniques.  Thus  it 
should  be  possible  to  locate  information  with  the  maxi¬ 
mum  facility  allowed  by  the  eurrent  technology.  The  in¬ 
dexing  is  most  important. 

Vehicle  3 — instructional  programs 

The  user's  handbook  is  intended  as  a  souree  of  facts 
for  experienced  computer  users.  By  contrast,  the  in- 
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struetional  programs  are  intended  to  edueate  a  class  of 
neophytes  up  to  a  state  of  being  experienced  users. 
Beeause  of  their  nature,  instructional  programs  will 
take  longer  to  develop  than  the  user’s  handbook.  When 
a  major  new  program  is  introduced,  the  handbook  will 
appear  first  and  the  instructional  program  will  lag  be¬ 
hind  a  certain  time.  This  time  delay  is  unfortunate  and 
will  be  minimized  by  computer  editing  techniques. 

The  instructional  programs  will  probably  not  be 
classic  programs  of  either  the  multiple-ehoiee-answer 
type  or  the  eonstrueted-answer  type.  For  the  most  part 
they  will  be  a  carefully  tested  text  whieh  will  involve 
many  exercises  to  be  done  by  the  student  on  the  com¬ 
puter.  These  may  or  may  not  involve  some  intervention 
on  the  part  of  a  human  teacher.  The  instructional  pro¬ 
grams  will  be  evaluated  by  actual  use  and  will  be  re¬ 
vised  as  necessary  in  order  to  achieve  a  given  level  of 
instruction.  The  programs  will  be  carefully  graded  as  to 
the  background  necessary  to  take  the  program.  Thus  if 
high  sehool  algebra  or  MAD  computer  programming 
or  MACROFAP  is  required  as  a  prerequisite  for  a 
given  program,  this  will  be  clearly  state. 

The  instructional  programs  will  be  stored  as  com¬ 
puter  files.  A  great  and  perhaps  primary  advantage  so 
achieved  is  to  allow  easy  and  general  editing  of  the 
files  to  keep  them  up  to  date.  Obsolescence  is  one  of 
the  most  difficult  problems  of  instructional  programs 
in  rapidly  changing  areas.  Computer  technology  may 
contribute  a  great  deal  to  instructional  program  tech¬ 
nology  in  this  area. 

Two  modes  of  administering  the  instructional  pro¬ 
grams  will  be  tried,  on  computer  consoles  and  as 
printed  paper  documents.  Thus  an  interesting  and  im¬ 
portant  comparison  between  console  distribution  and 
paper  document  distribution  is  inherent  in  this  vehicle. 

Vehicle  4 — Miss  Know-It-All 

This  vehicle  will  definitely  not  be  on  a  computer  file. 
It  consists  of  a  person  who  is  a  combined  reference 
librarian,  instructress,  psychotherapist,  and  program 
counsellor.  Such  a  position  might  appear  hard  to  fill.  I 
think  it  will  be  possible  to  find  a  number  of  women 
applicants  who  have  the  highest  abilities  but  whose  per¬ 
sonal  research  has  been  interrupted  through  matrimony. 
These  will  still  have  sufficient  time  and  interest  to  do 
a  job  of  this  nature.  Such  people  will  have  a  number  of 
functions.  They  will  be  a  source  of  information  about 
documents  and  fact  retrieval.  Here  we  should  be  able 
to  get  a  very  good  comparison  between  the  annotated 
catalog  and  a  person  for  direction  on  where  to  find  a 
particular  piece  of  information.  It  has  often  been  stated 
that  the  right  person  is  a  much  better  guide  than  any 
catalog,  but  this  assertion  has  never  been  really  tested, 
and  a  comparison  could  be  made  in  this  environment. 


They  will  be  a  guide  to  students  using  the  instructional 
programs.  Here  part  of  their  function  would  simply  be 
to  see  that  the  students  have  the  right  background  and 
use  the  correct  instructional  programs  for  the  educa¬ 
tion  they  need.  Another  part  will  be  to  encourage  the 
students  so  that  they  do  not  extinguish  in  going  through 
a  given  program.  Still  another  is  to  provide  some  human 
instruction  and  to  take  care  of  the  situations  whieh  the 
written  programs  do  not  accommodate.  They  will  pro¬ 
vide  program  counselling.  The  program  counsellor  ean 
suggest  ways  in  which  various  programs  should  be 
written,  ean  suggest  existing  programs  or  subroutines 
for  doing  particular  jobs  and  ean  provide  some  assist¬ 
ance  or  at  least  moral  support  in  debugging  programs. 

In  general,  Miss  Know-It-All  will  provide  a  human 
source  of  information  which  can  be  compared  with  the 
mechanization  which  we  arc  imposing  on  our  other 
vehicles. 

Vehicle  5 — computer  programs 

Little  need  be  said  about  computer  programs.  They 
will  be  stored  on  computer  files  and  will  be  the  basic 
information  in  the  system.  Some  will  be  almost  human¬ 
ly  readable  and  some  will  be  very  difficult  to  read, 
depending  on  the  language  in  whieh  they  are  written. 
In  many  or  even  most  eases  it  will  be  desirable  to  have 
adequate  annotation  so  that  a  person  ean  figure  out 
what  the  program  does.  It  is  perhaps  useful  to  divide 
the  programs  into  three  elasses: 

•  Public  programs.  These  programs  will  have  very 
rigid  standards  of  documentation,  some  of  whieh 
have  already  been  mentioned  in  connection  with 
the  user’s  handbook.  The  program  itself,  in  addi¬ 
tion  to  the  handbook,  will  be  required  to  have  a 
very  complete  and  a  very  high  standard  of  annota¬ 
tion.  Such  annotation  must  be  provided  with  the 
original  program. 

•  Scmipublie  programs.  These  programs  will  be 
written  by  various  individuals  and  made  available 
to  the  publie,  somewhat  at  the  public’s  own  risk. 
They  would  correspond  to  SHARE  programs. 
The  same  standards  of  documentation  and  annota¬ 
tion  will  be  applied  to  these  as  arc  applied  to  the 
publie  programs.  However,  the  enforcement  of 
these  standards  will  be  more  lax  in  that  only 
psychological  pressure  will  be  applied  to  achieve 
conformity. 

•  Private  programs.  These  are  programs  intended 
to  be  used  by  one  or  a  very  few  people.  Here, 
although  annotation  and  documentation  still  seem 
desirable,  it  will  be  up  to  the  user  to  do  what 
he  wishes.  Different  people  vary  in  the  degree 
they  find  most  advantageous  and  in  addition  the 
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originator  will  be  the  primary  person  to  suffer 
from  any  sins  of  omission. 

Vehicle  6 — definitive  summaries  of 

the  state  of  the  computation  art 

These  summaries  and  reports  will  be  formal  pub¬ 
lished  papers  which  are  requested  and  paid  for  by  a 
committee  of  experts  which  is  formed  for  this  express 
purpose.  The  committee  will  meet  for  one  week  every 
six  months.  (In  the  Summer  the  meeting  will  be  held  at 
the  National  Academy  establishment  at  Woods  Hole 
and  in  the  Winter  the  meeting  will  be  held  alternately 
in  Honolulu  and  Bermuda.  Occasionally  a  meeting  at 
Lake  Como  will  be  held  to  incorporate  European  opin¬ 
ions. )  The  committee  will  review  progress  over  the  last 
six  months,  will  commission  the  writing  of  papers  to 
describe  this  progress,  and  will  review  papers  already 
written.  The  material  already  written  may  either  be 
published,  or  may  be  sent  back  to  the  author  for  revi¬ 
sion,  or  may  be  rejected  and  sent  to  another  author 
for  rewriting.  The  highest  standards  of  importance, 
technical  correctness,  clarity,  and  tutorial  informative¬ 
ness  will  be  maintained  in  these  papers.  Being  asked  to 
write  such  a  paper  will  be  considered  a  high  honor,  one 
which  is  weighted  heavily  by  academic  committees  in 
charge  of  promotion.  Authors  will  be  paid  well  for 
their  writing  and  encouraged  to  take  a  sabbatical  and 
go  to  some  nice  isolated  place  in  order  to  do  an  un¬ 
disturbed  and  excellent  job. 

These  definitive  papers  will  be  carefully  referenced, 
cited,  abstracted,  and  indexed.  As  the  state  of  the  in¬ 
formation  retrieval  art  progresses  they  may  well  be 
written  in  a  form  to  facilitate  the  machine-retrieval  of 
their  facts.  This  may  necessitate  annotating  the  docu¬ 
ments  with  special  machine-readable  code  in  order  to 
facilitate  “understanding”  of  the  material  by  a  comput¬ 
ing  machine.  The  documents  will  be  of  such  value  that 
the  additional  effort  to  make  them  machine-understand¬ 
able  will  be  well  worthwhile.  In  any  case,  the  documents 
will  be  in  machine-readable  form  and  will  be  available 
on  computer  files  as  well  as  in  published  version.  They 
will  form  part  of  the  world’s  permanent  computer 
literature. 

Vehicle  7 — meetings  of  computer  user  society 

A  society  of  computer  users  and  makers  will  meet 
for  a  few  days  every  six  months.  These  meetings  will 
be  patterned  after  the  present  meetings  of  the  Acoustical 
Society  of  America.  Oral  papers  with  published  ab¬ 
stracts  will  be  presented.  Every  effort  will  be  made  to 
achieve  the  maximum  of  oral  communication  between 
the  attendees  at  the  meeting.  No  attempt  at  written 
publication  of  the  entire  papers  will  be  made.  The  at¬ 


tendees  will  be  free  to  publish  the  works  which  they 
feel  are  worthwhile  in  other  standard  computer  publica¬ 
tions.  Effective  oral  communication  will  be  facilitated 
by  the  newspaper  (Vehicle  1 )  which  will  give  all  the 
participants  a  common  background  of  information  as 
well  as  by  the  various  other  vehicles  to  which  they  have 
common  access.  Participants  will  be  encouraged  to  be 
highly  critical  of  the  papers,  discussion  after  presenta¬ 
tion  will  be  encouraged,  and  in  general  all  forms  of 
oral  communication  will  be  pushed.  The  most  damning 
criticism  of  a  paper  will  be  that  it  is  presented  in  an 
unintelligible  way;  the  second  that  it  contains  nothing 
of  interest. 

CONCLUSION 

The  seven  vehicles  described  above  are  in  themselves 
not  new.  However,  putting  them  together  into  one 
integrated  system  to  handle  the  communications  in  a 
particular  area  is  a  new  experiment.  The  vehicles  which 
have  been  described  are  designed  to  provide  rapid 
written  communication  of  new  items,  to  provide  de¬ 
finitive  information  about  the  current  technology,  to 
provide  instruction  in  the  current  art,  to  provide  a  hu¬ 
man  source  of  reference  for  all  information,  to  provide 
the  actual  technical  vehicles  (which  in  this  case  are 
computer  programs),  to  provide  definitive  information 
in  the  form  of  a  permanent  record  of  the  developments 
of  the  field  and  to  provide  a  vehicle  for  oral  communi¬ 
cation  in  a  reasonable  environment. 

The  particular  vehicles  and  the  way  in  which  they 
have  been  discussed  is  pertinent  to  computer  science. 
However,  most  of  the  needs  of  computer  science  apply 
to  other  fields  with  more  or  less  emphasis  depending 
upon  the  particular  field.  Thus,  though  the  vehicles 
described  may  differ  a  bit  for  another  field,  a  similar 
set  of  vehicles  would  be  involved.  Certainly  the  areas 
of  rapid  news  communication,  of  a  handbook  of  facts, 
of  instruction,  of  oral  communication  and  of  definitive 
papers  are  involved  in  almost  every  field. 

With  the  exception  of  Miss  Know-It-All  and  the  oral 
meetings,  all  the  vehicles  are  suitable  for  distribution 
either  on  computer  files  or  as  pieces  of  paper,  and  I 
have  frequently  pointed  out  possible  comparisons  of 
these  media. 

How  can  an  improvement  in  our  technical  commu¬ 
nications  be  initiated?  Clearly  this  is  a  most  difficult 
question,  but  two  thoughts  seem  pertinent.  First,  it  will 
be  much  easier  to  start  with  a  new  system  and  high 
standards  in  a  new  area  than  to  improve  an  existing 
area  which  already  has  established  habits.  Thus,  com¬ 
puter  science  seems  a  hopeful  starting  point.  Second, 
the  importance  of  quality  and  communication  in  our 
technical  literature  must  be  continually  emphasized.  The 
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technical  experts  in  a  given  field  must  be  reminded  that 
they  are  responsible  for  the  quality  of  their  literature. 
Nothing  any  library  system,  either  regular  or  com¬ 
puterized,  can  do  will  help  nearly  as  much  as  careful 
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writing  by  authors.  In  the  final  analysis,  purification 
of  the  literature  will  be  determined  by  the  attitudes  of 
the  members  of  the  technology.  They  will  create  their 
heaven  or  hell. 
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Introductory  Remarks 

Gordon  T.  Gould,  Jr.,  Major  General,  USAF 
Headquarters  USAF 
Washington,  D.  C. 

A  decade  of  pioneering  work  on  Military  Command 
Information  Systems  has  equipped  the  major  military 
echelons  with  their  first  generation  of  computer 
assisted  command  and  control  systems. 

The  current  complex  of  individual  first  generation 
systems  were  largely  designed  and  implemented 
independently  and  unilaterally  as  the  requirement 
evolved.  As  a  result,  today,  military  command  and 
control  systems,  on  a  world-wide  basis,  use  more  than 
20  different  kinds  of  computer  hardware  produced  by 
more  than  four  different  manufacturers.  The  pro¬ 
grams  for  these  computers  are  written  in  more  than  20 
different  programming  languages.  At  least  10  of  these 
programming  languages  are  each  used  in  only  one  sys¬ 
tem.  Three  of  these  languages  are  used  in  seven  or 
more  systems. 

Certain  factors  therefore  become  apparent  to  man¬ 
agers  concerned  with  military  command  and  control 
—  whether  they  be  in  the  military  or  in  industry.  Cost 


effectiveness  and  operational  effectiveness,  on  a 
world-wide  basis,  will  be  key  watch  words  as  we 
implement  the  next  generation  of  military  command 
and  control  systems.  Compatibility,  commonality,  and 
exchangeability  will  be  emphasized  up  to  but  not  be¬ 
yond  the  point  where  they  degrade  the  user’s  military 
operational  capability. 

Looking  at  military  command  and  control  with  a 
system  point  of  view,  certain  judgments  can  be  drawn 
with  respect  to  exploiting  the  computer.  Remarkable 
progress  is  being  made  in  advancing  the  hardware 
state  of  the  art.  However,  a  lesser  rate  of  advance  is 
being  made  in  other  systems  aspects.  Also,  these 
aspects  are  much  more  difficult  to  quantify  than  in 
the  case  of  the  hardware.  Examples  of  the  more 
difficult  system  aspects  are:  definition  of  the  user’s 
real  needs;  responsibilities  of  the  system  engineer; 
relationships  between  the  user  and  the  system  engi¬ 
neer;  retrieval  and  display  of  information  to  the 
decision  maker;  and,  design  and  production  of  soft¬ 
ware. 

The  papers  in  this  section  discuss  these  aspects  of 
exploiting  data  processing  capabilities  in  military 
command  and  control. 


191 


System  engineering  experience  with 
automated  command  and  control  systems 


by  David  R.  Israel 
The  MITRE  Corporation 
Bedford,  Massachusetts 

INTRODUCTION 

This  paper  addresses  itself  to  some  of  the  key  problems 
arising  in  the  design,  engineering,  and  implementation 
of  automated  military  command  and  eontrol  systems 
in  which  real-time  data  processing  plays  a  central  role. 
More  specifically,  I  have  attempted  to  extract  and  sum¬ 
marize  the  experience  obtained  during  roughly  the  first 
decade  of  the  acquisition  and  use  of  Air  Force  com¬ 
mand  and  eontrol  systems. 

This  “decade”  may  be  roughly  defined  as  having 
started  in  the  early  1950's  with  the  initiation  of  efforts 
on  the  SAGE  (416L)  air  defense  system  for  Con¬ 
tinental  U.S.,  and  closed  in  196b  with  a  turnover  of 
the  NORAD  Combat  Operations  Center  System  (425 L). 
In  between,  we  have  seen  completion  of  the  efforts 
on  the  SACCS  (465L)  system  for  SAC,  the  412L  air 
defense  system  for  USAFE,  and  the  BUIC  (41 6M) 
back-up  system  for  SAGE;  we  have  also  obtained  sig¬ 
nificant  experience  from  the  on-going  efforts  on  the 
Headquarters  USAF  Command  System  (473L)  and 
USSTRICOM  Command  and  Control  System  (492L). 

While  it  may  be  somewhat  premature  to  judge  the 
results  of  what  is  essentially  the  first  generation  of  Air 
Foree  command  and  control  systems,  it  is  clear  that 
at  least  the  control  systems  differ  significantly  from  the 
more  conventional  application  of  computers  to  scien¬ 
tific  problems  or  business  data  processing,  and  that 
both  types  of  systems  have  represented  a  significant  in¬ 
crease  in  technical  complexity  over  previous  engineering 
experience  with  electronic  systems  and  equipments.  It 
is  not  surprising,  then,  that  successful  system  develop¬ 
ment  and  implementation — as  measured  in  terms  of 
expected  cost,  schedules,  and  performance — has,  to 
date  at  least,  been  more  the  exception  than  the  rule. 

The  material  that  follows  is  written  from  a  specialized 
point  of  view;  namely,  that  of  a  supplier  of  technical 
advice,  assistance,  and  system  engineering  support  to 
the  Electronic  Systems  Division  of  the  Air  Force  Sys¬ 
tems  Command,  whieh  has  been  primarily  responsible 


for  the  aequistion  of  the  systems  mentioned.  It  is  not 
necessarily  the  viewpoint  of  the  Electronic  Systems 
Division,  nor  that  of  the  users  or  operators,  nor  that 
of  the  industrial  contractors  or  suppliers  of  system 
components.  I  have  tried  to  make  it  as  technically  ob¬ 
jective  as  possible,  but  one  must  expect  that  there  are 
bounds  to  a  participant's  point  of  view. 

This  paper,  however,  is  written  for  a  somewhat 
broader  audience:  those  in  the  military  and  government 
as  well  as  those  outside  the  government  who  may  be 
concerned  with  the  acquisition  of  similar  systems,  for 
it  is  elcar  that  many  of  the  problems  encountered  are 
not  unique  to  military  applications,  but  arc  representa¬ 
tive  of  any  use  of  real-time  information  processing 
systems. 

Accordingly,  and  with  such  a  broader  audience  in 
mind,  I  have  refrained  from  using  much  of  the  specific 
Air  Force  terminology  and  from  referring  to  the  per¬ 
tinent  Air  Force  procedures,  regulations,  and  formats 
which  have  come  into  being  over  the  past  decade. 
Rather,  the  attempt  has  been  made  to  provide  a  dis¬ 
cussion  independent  of  reference  to  specific  Air  Foree 
jargon,  procedures,  regulations  or  usage. 

In  the  sections  which  follow,  frequent  use  is  made  of 
the  terms  system  design  and  system  engineering.  I  will 
use  system  engineering  as  the  broader  activity  ineluding 
such  functions  as  design,  technical  support  of  procure¬ 
ment  and  production,  and  testing.  A  major  point,  if 
not  the  theme  of  this  paper,  is  that  system  engineering 
must  include  a  host  of  considerations  which  arc  not 
strietly  technical  if  successful  system  development  and 
operation  is  the  goal.  System  design  and  engineering 
must  attempt  to  reach  an  appropriate  balance  or  com¬ 
promise  among  the  factors  of  operational  requirements, 
teehnologieal  capabilities,  costs,  and  schedules. 

The  system  engineering  activity  must  take  an  ex¬ 
tremely  broad  point  of  view.  Its  scope  must  cover  the 
long  time  span  stretching  from  initial  concept  to  ul¬ 
timate  operation,  and  even  beyond.  Even  during  ac- 
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quisition,  these  systems  do  not  remain  statie,  but  evolve 
to  meet  new  requirements.  Systems  engineering  con¬ 
siderations,  as  well  as  encompassing  a  long  time  span, 
must  also  cover  a  diversity  of  topics.  It  is  unwise,  if 
not  dangerous,  to  consider  only  the  equipments  and 
computer  programs  directly  related  to  the  operational 
functions.  Proper  system  engineering  will  consider  sueh 
varied  topies  as  availability,  training,  and  exercising 
of  operators;  operational  procedures;  provisions  for 
the  maintenance  of  hardware  and  computer  programs; 
the  availability  and  operating  conditions  of  government- 
furnished  equipments;  communications  integration  with 
other  systems;  civil  engineering  construction  of  build¬ 
ings  and  related  facilities,  and  so  on. 

Beyond  this,  it  is  increasingly  evident  that  successful 
system  development  is  heavily  dependent  upon  the 
organization  and  management  techniques  used  to  guide 
the  system  from  concept  to  field  operation. 

It  is  in  this  broad  eontext  of  topies  stretehing  over 
a  long  time  frame  that  1  will  discuss  some  of  the  design 
and  engineering  problems  of  eommand  and  control 
systems.  Within  the  constraints  noted  above,  instances 
from  the  past  10  years  of  experience  will  be  cited. 

In  the  next  scetion,  some  of  the  principal  characteris¬ 
tics  of  eommand  and  control  systems  will  be  noted. 
Section  II  presents  comments  on  pertinent  matters  of 
organization  and  management  of  the  system  engineering 
and  acquisition  activity.  Seetion  III  diseusscs  some  of 
the  initial  considerations  whieh  should  guide  the  over¬ 
all  system  design;  Seetion  IV  is  eoneerned  with  the 
proeess  and  techniques  of  the  design  effort.  Detailed 
aspects  of  hardware,  software,  and  testwarc  design 
follow  in  succeeding  sections. 

As  a  final  introductory  remark,  let  me  say  that  there 
arc  no  known  magic  formulas,  no  known  optimum 
procedures  or  techniques,  nor  any  10  easy  lessons  to 
successful  system  design  and  engineering.  My  principal 
intentions  are  only  to  point  out  items  which  should  be 
considered  and  to  illuminate  possible  pitfalls.  Much  of 
what  is  presented  may  seem  like  little  more  than  or¬ 
dinary  common  sense.  If  so,  I  can  only  wish  that  our 
foresight  had  been  as  good  as  our  present  hindsight. 
We  have  learned  much  about  system  engineering  over 
the  past  decade;  we  have  learned  much  less  about  how 
to  document  or  apply  this  experience;  and  some  things 
we  have  relearned.  I  am  quite  eonseious  of  the  difficulty 
of  transferring  such  experience,  and  only  hope  that  the 
material  which  follows  ean  in  some  way  be  of  benefit 
to  the  engineers  and  designers  of  future  systems. 

/.  System  characteristics 

I  do  not  believe  it  is  possible — or  necessarily  useful — 
to  establish  a  completely  satisfactory  definition  of  this 


elass  of  systems.  Instead,  and  as  a  means  of  estab¬ 
lishing  a  basis  for  the  subsequent  discussion,  let  me 
summarize  some  of  the  principal  characteristics  of 
these  military  data  processing  systems: 

•  They  are  generally  large,  as  measured  in  terms 
of  equipment  or  geography,  and  usually  very 

eostly. 

•  They  are  typieally  made  up  of  many  elements  or 
subsystems — computers,  displays,  communica¬ 
tions,  personnel,  computer  programs,  data  basis, 
input  data  sources,  and  output  data  users — all 
of  which  must  be  earefully  integrated. 

•  They  proeess  large  amounts  of  raw  data  from 
diverse  sources,  converting  this  to  summarized 
information  for  men  or  other  machines  or  systems. 

•  They  must  perform  the  data  processing  in  “real- 
time”  in  order  that  the  system  response  keeps 
paee  with  incoming  data  and  required  outputs, 
and  they  must  be  available  on  a  continuous, 
around-thc-elock,  basis. 

•  While  they  normally  operate  well  below  their 
design  eapaeity,  a  eapaeity  whieh  may  never  be 
required  exeept  in  wartime,  they  must  operate 
continuously  at  a  high  level  of  readiness. 

•  They  usually  replaee  an  existing  manual  sys¬ 
tem  and,  in  turn,  must  operate  as  a  part  of  some 
larger  military  system. 

•  Despite  the  introduction  of  automatic  data  pro¬ 
cessing,  human  operators  have  a  predominant 
role  in  the  system  operation. 

•  They  have  generally  been  designed  and  imple¬ 
mented  by  one  organization  (an  acquisition  agen- 
ey)  for  use  by  another  (the  operator  or  user). 

•  Because  of  their  size  and  importance,  their  de¬ 
velopment  is  usually  marked  by  a  large  number 
of  approval  levels  and  coordination  channels. 

•  They  take  a  significant  time  to  aequire,  during 
which  time  both  the  requirements  and  the  avail¬ 
able  technology  may  ehange. 

•  They  must  change  and  evolve  during  their  op¬ 
erational  lifetime  to  meet  new  requirements  and 
situations. 

•  They  usually  defy  an  a  priori ,  detailed,  quantita¬ 
tive  specification  of  their  performance  bceause 
of  their  complexity  and  the  unpredictability  of  the 
conditions  under  whieh  they  operate. 

It  is  similarly  difficult  to  define  with  precision  the 
differences  between  command  systems  and  control  sys¬ 
tems.  In  terms  of  physical  composition,  the  two  types 
of  systems  have  been  quite  similar,  both  involving 
medium-  to  large-sealc  data  processors,  operator  con¬ 
soles,  and  communication  tics.  From  a  pure  hardware 
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engineering  standpoint,  the  problems  involved  in  imple¬ 
menting  them  are,  therefore,  quite  similar. 

The  basic  differences  seem  to  arise  primarily  be¬ 
cause  of  the  relationships  of  the  system  to  the  job  that  it 
is  supporting.  Control  systems  have  tended  to  be  very 
much  in  line  with  the  job,  with  the  individual  job  de¬ 
scriptions  closely  coupled  to  the  manner  in  which  the 
operators  interface  with  the  system.  In  the  case  of  com¬ 
mand  systems,  we  currently  find  more  of  an  off-line 
relationship  between  the  system  and  the  military  job 
that  it  is  supporting.  Many  command  system  user  per¬ 
sonnel  do  not  accomplish  their  primary'  job  by  sitting  at 
operator  consoles  throughout  the  day,  but  rather,  turn 
occasionally  to  the  system  to  obtain  an  output  that  pro¬ 
vides  an  assist  in  the  accomplishment  of  a  primary  task. 
In  other  words,  the  control  systems  have  incorporated 
a  much  higher  percentage  of  the  tasks  involved  in  ac¬ 
complishing  specified  military  jobs  than  have  the  com¬ 
mand  systems  that  have  been  provided  to  date.  It  is 
not  clear  that  this  must  be  the  case,  and  greater  effort 
needs  to  be  made  to  increase  the  percentage  of  the 
military  jobs  directly  supported  by  the  command  sys¬ 
tems. 

This  loose  relationship  between  the  command  sys¬ 
tems  and  the  military  job  that  they  are  supporting  has 
process.  The  functional  requirements  for  control  sys¬ 
tems  tend  to  be  more  definitive  than  those  for  com- 
in  turn  had  considerable  impact  on  the  acquisition 
mand  systems.  This  fact  then  makes  the  translation 
from  requirements  to  design  more  difficult  for  com¬ 
mand  systems.  In  this  situation,  the  designer  often  has 
little  to  draw  on  from  his  own  background  and  is  de¬ 
pendent  on  the  user  to  supply  the  detailed  description 
of  the  job  which  the  system  is  supporting.  This  sort  of 
description  is  seldom  available  in  a  documented  form, 
and  the  designer  may  be  forced  to  ‘‘move  in”  with  the 
user  to  gain  sufficient  knowledge  of  the  job  to  bring  the 
technology  to  bear  on  the  problem. 

The  problems  of  simulating  the  system  environment 
during  the  developmental  stages  arc  much  more  difficult 
in  the  case  of  command  systems.  No  one  has  yet  suc¬ 
cessfully  simulated  a  command  system  environment.  On 
the  other  hand,  the  environment  for  such  control  sys¬ 
tems  as  SAGE  and  BUIC  has  been  simulated  quite 
realistically  by  simply  tying  the  command  centers  into 
the  surveillance  and  communication  networks,  or  by 
running  the  system  against  recorded  surveillance  data 
and  simulated  interceptor  actions.  The  test  and  evalu¬ 
ation  of  control  systems,  then,  has  been  more  straight¬ 
forward,  albeit  still  a  difficult  task. 

The  problems  of  phaseover  to  operational  status  of 
the  two  types  of  systems  arc  quite  different.  In  the 
case  of  control  systems,  the  user  seems  to  be  more  will¬ 


ing  to  adapt  to  the  system,  even  to  the  extent  of  revising 
his  organizational  structure.  This  has  not  been  true  in 
the  case  of  command  systems. 

II.  Organization  and  management 

In  keeping  with  the  stated  broad  perspective,  I  will 
next  discuss  some  pertinent  considerations  relating  to 
organization  and  responsibilities,  operational  inputs, 
and  procurement.  This  discussion  is  not  intended  to  be 
either  definitive  or  complete  in  its  treatment  of  these 
nontechnical  matters.  Rather,  I  have  only  attempted  to 
stress  those  points  which  are  highlighted  by  command 
and  control  experience. 

Project  organization 

It  is  generally  agreed  today  that  if  one  is  to  acquire 
a  large  new  system,  a  strong,  centralized,  knowledge¬ 
able  project  organization  should  be  established  by  and 
within  the  government  agency  with  the  primary  respon¬ 
sibility  for  system  acquisition  and  development.  This 
seems  to  be  necessary  regardless  of  the  type  of  con¬ 
tractual  responsibilities  and  arrangements  made:  prime 
system  contractors,  associate  contractors,  or  equip¬ 
ment  suppliers. 

In  the  material  that  follows,  it  will  be  assumed  that 
project  organization  is  synonymous  with  acquisition 
agency  and  with  procurement  agency. 

This  project  organization  should  have  a  broad  charter, 
both  in  scope  and  in  time.  It  must  consider  the  relation¬ 
ship  of  such  problems  as  construction  and  facilities, 
training,  maintenance,  and  exercising  to  the  overall 
design,  since  they  are  likely  to  exert  a  strong  influence 
on  the  system  and  its  eventual  performance.  It  should 
be  charged  with  coordination  of  all  aspects  of  the  de¬ 
sign.  It  should  review  and  approve  all  schedules,  mon¬ 
itor  progress,  provide  a  central  and  common  focus 
for  all  system  decisions,  and  exccrisc  the  necessary  re¬ 
source  management. 

This  project  organization  should  have  a  single,  en¬ 
thusiastic,  strong-minded  leader.  It  should  not  be  headed 
by  a  committee  that  does  not  have  requisite  authority  or 
that  can  arrive  at  decisions  only  by  finding  the  least 
common  multiple.  The  project  organization  should  have 
or  be  directly  supported  by  technical  competence  equal 
to  the  job  at  hand.  As  discussed  below,  this  may  vary 
with  the  contractual/procuremcnt  arrangements,  but 
in  no  case  should  the  project  organization  find  itself 
incapable  of  dealing  on  an  adequate  technical  basis 
with  the  contractors. 

User  inputs  and  support  are  equally  important,  and 
this  is  also  elaborated  upon  below. 

The  location  of  the  project  organization  is  an  impor¬ 
tant  consideration.  Collocation  with  the  user  seems  to 
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be  desirable  especially  in  the  case  of  command  systems, 
but  may  not  be  practical  since  the  project  organization 
may  in  turn  be  part  of  some  larger  organization  han¬ 
ding  several  system  acquisitions.  In  such  cases,  the  in¬ 
tegrity  of  this  broader  organization  may  be  important 
for  administrative  and  other  reasons,  including  common 
use  of  certain  specialists  in  various  areas.  Location  of 
a  system  test  bed  or  overhead  facility  (see  Section  IV) 
may  also  be  a  factor.  Where  collocation  with  the  user 
is  possible,  a  joint  organization  including  user  personnel 
has  been  found  to  be  a  very  effective  way  of  guaran¬ 
teeing  the  user  inputs  discussed  below. 

Because  the  new  system  may  very  likely  use  part  or  all 
of  the  facilities  and  communications  of  an  existing  (man¬ 
ual)  system,  problems  have  arisen  in  the  control  of  the 
facility  and  communication  contractors.(Even  for  new 
communications,  the  use  of  user  operating  and  mainte¬ 
nance  funds  for  rental  of  communications  has  removed 
much  of  the  authority  of  the  project  organization.)  In 
some  cases,  the  project  organization  has  had  little  or 
no  direct  authority  over  these  contractors.  This  situa¬ 
tion,  coupled  with  tortuous  coordination  channels,  has 
caused  serious  problems  in  obtaining  adequate  response 
to  communication  and  facility  requirements.  For  ex¬ 
ample,  in  the  case  of  required  modifications  to  the 
Cheyenne  Mountain  Complex  Facility  of  the  425L 
system,  requests  had  to  pass  from  the  project  office  to 
the  user,  then  through  several  Air  Force  levels  to  the 
Corps  of  Engineers  to  the  facility  contractor.  In  the 
case  of  communications,  requests  were  forwarded 
through  the  user  to  the  telephone  companies  concerned, 
and,  in  some  cases,  on  to  subcontractors  of  the  com¬ 
panies.  This  serial  coordination  is,  of  necessity,  time 
consuming,  even  assuming  completely  cooperative  at¬ 
titudes  on  the  part  of  all  parties.  Adequate  arrangements 
concerning  control  and  direction  of  the  facility  and 
communication  contractors,  then,  should  be  attempted. 

Finally,  the  project  nature  of  the  organization  should 
be  stressed.  It  should  be  organized  for  a  specific  pur¬ 
pose  and  given  the  necessary  authority  and  responsi¬ 
bility.  It  is  the  antithesis  of  the  stable  technical 
organization  formed  along  conventional  lines.  An  at¬ 
tempt  to  split  up  the  acquisition  and  engineering  jobs 
to  satisfy  some  existing  functional  organization  should 
be  avoided,  since  it  is  likely  to  jeopardize  the  entire 
endeavor  from  the  outset. 

Operational  inputs 

The  arrangements  for  the  project  organization  should 
ensure  early,  continuous,  and  active  representation  from 
the  ultimate  using  and  operating  commands*  This  is 

*In  some  situations,  the  using  and  operating  commands  may 
not  be  the  same.  This  distinction  will  be  ignored  here,  and  the 
terms  user  and  operator  will  be  used  interchangeably. 


a  requirement  in  the  design  process  for  the  following 
reasons:  first,  it  will  provide  a  direct  channel  of  infor¬ 
mation  concerning  changing  requirements;  second,  it 
will  provide  the  necessary  contact  with  field  operating 
problems;  and  third,  it  will  provide  mutual  appreciation 
of  problems  by  the  developer  and  user. 

There  is  a  strong  and  very  dangerous  tendency  for 
an  engineer  to  design  the  system  for  himself  to  operate, 
and  generally  to  operate  only  for  short  periods  of  time 
and  at  high,  or  at  least  operationally  interesting,  load 
levels.  The  actual  operators,  on  the  other  hand,  are  less 
sophisticated  technically  and  usually  must  operate  the 
system  day  in  and  day  out  for  long  periods  at  very  low 
load  levels — levels  at  which  the  automatic  system  gen¬ 
erally  is  not  required. 

To  put  it  in  somewhat  cruder  terms,  the  engineer 
tends  to  design  a  sophisticated  system  which  is  difficult 
to  operate  or  which  only  he  can  operate.  On  the  other 
hand,  the  operator  tends  to  design  by  adding  a  num¬ 
ber  of  small  fixes  to  isolated  problems.  This  approach, 
in  turn,  results  in  less  capability  than  might  be  achieved 
for  the  same  investment. 

The  solution  lies  in  close  participation  by  the  using 
command  throughout  the  design  and  acquisition  phases. 
The  participators  must  be  carefully  chosen  personnel, 
and  their  inputs  must  be  controlled  so  that  the  require¬ 
ment  does  not  become  merely  a  “wish  book”  which 
cannot  be  met  within  the  constraints  of  the  allocated 
resources.  The  participation  should  be  continuous  if 
waste  is  to  be  avoided.  Confrontation  between  the  op¬ 
erator  and  developer  at  only  six-month  or  longer  in¬ 
tervals  leads  to  misunderstandings,  unnecessary  effort, 
rework,  and  often  unsatisfactory  compromises  resulting 
from  tradeoffs  occurring  too  late  in  time.  The  operator 
should  be  required  to  concur  on  all  operational  design 
features,  particularly  those  relating  to  operational  per¬ 
sonnel  and  man-machinc  relationships. 

The  representatives  from  the  using  command  should 
include  both  planners  and  operators,  with  the  balance 
gradually  shifting  to  the  latter  as  the  project  matures. 
The  necessity  to  keep  key  personnel  restricted  to  their 
day-to-day  operational  jobs  may  cause  difficulty  in 
achieving  proper  user  representation.  The  expected  rate 
of  system  development  should  take  account  of  such 
difficulties. 

Direct  representation  from  the  using  command  can 
have  two  bonus  effects.  First,  it  provides  for  a  direct 
association  of  the  operating  command  with  the  system 
development  process,  and  thus  gives  the  operating  com¬ 
mand  an  opportunity  to  be  party  to  design  and  develop¬ 
ment  decisions  as  (hey  are  being  made.  In  this  way,  the 
user  gains  a  helpful  understanding  of  the  problems  as¬ 
sociated  with  system  development.  And  secondly,  it 
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may  help  to  alert  and  prepare  the  operating  command 
for  actually  using  the  new  system.  This  command  is 
generally  overcommitted  to  the  operation  of  an  existing 
system,  and  finds  it  difficult  to  prepare  for  operation  of 
the  new  system.  If  the  change  is  too  drastic,  the  useful 
operational  data  of  the  system  is  delayed  or  the  system 
may  never  be  used  if  the  operator  is  not  “prepared.” 

A  problem  which  has  arisen  with  command  systems 
is  that  no  single  or  authoritative  “user”  or  focal  point 
in  the  command  has  been  named,  with  the  result  that 
the  whole  gamut  of  the  staff  agencies  must  be  dealt 
with.  This,  naturally,  results  in  a  proliferation  of  sys¬ 
tem  requirements.  The  need,  then,  is  to  find  the  user; 
when  there  are  several,  someone  must  be  named  as  the 
authoritative  focal  point,  and  preferably,  this  should 
be  done  by  providing  him  with  appropriate  and  special 
authority  for  this  purpose. 

System  engineering  capability 

It  is  essential  that  a  capability  to  accomplish  the 
system  engineering  and  technical  services  of  the  type 
described  throughout  this  paper  be  provided  to  the 
project  organization.  Beyond  its  more  conventional 
design  and  engineering  functions,  this  system  engineer¬ 
ing  capability  must  also  attempt  to  safeguard: 

The  user's  interest  by — 

•  Ensuring  that  design  specifications  arc  responsive 
to  operational  needs. 

•  Ensuring  that  test  criteria  measure  operational 
capability. 

•  Translating  contractor  design  decisions  into 
operational  impact. 

•  Injecting  new  techniques,  as  feasible  within  the 
scope  of  the  contract. 

The  contractor's  interest  by — 

•  Ensuring  that  specifications  and  user-requested 
changes  arc  reasonable. 

•  Ensuring  that  there  is  a  good  technical  under¬ 
standing  of  the  contractor’s  problems. 

The  procurement  agency’s  interest  by — 

•  Ensuring  that  specifications  represent  a  best 
compromise  between  user  desires  and  available 
funds. 

•  Ensuring  that  the  efforts  of  several  contractors 
arc  appropriately  integrated. 

•  Advising  on  impact  of  contractor  approaches 
and  design  decisions. 

•  Monitoring  schedules  from  the  point  of  view  of 
contractor  actions  rather  than  reports. 

Clearly  this  activity  is  not  a  one-time  process  which 
can  be  completed  early  in  the  acquisition  period.  The 
system  engineering  effort  which  is  addressed  here  is 
that  which  is  concerned  with  the  entire  definition  and 
acquisition  phases  and  proceeds  from  preparation  of 


definitive  performance  or  design  specifications  through 
procurement,  installation  and  test  activities  to  the  op¬ 
erational  phase  of  a  program.  Properly  executed,  it  is 
a  continuous  process.  Where  adequate  attention  has 
not  been  given  to  such  efforts,  serious  problems  of  de¬ 
lays,  cost  overruns,  or  unsatisfactory  performance  have 
resulted. 

The  system  engineering  efforts  can  be  obtained  from 
technical  personnel  on  the  staff  of  the  project  organ¬ 
ization,  can  be  contracted  from  organizations  specializ¬ 
ing  in  such  services,  or  can  be  obtained  from  the 
industrial  contractors  participating  in  the  program. 
These  are  not  exclusive  approaches;  the  selection  or 
choice  generally  depends  upon  the  existing  situation, 
and  combinations  of  the  three  possibilities  have  been 
used  successfully. 

Selection  of  the  procurement  method  obviously  af¬ 
fects  the  choice  of  the  size  and  source  of  the  system 
engineering  support.  However,  regardless  of  the  num¬ 
ber  or  roles  of  industrial  contractors  in  a  system  ac¬ 
quisition,  a  project  organization  generally  has  need  of 
additional  technical  services.  Many  technical  decisions 
must  be  made  after  contracting,  regardless  of  the  type 
of  contract.  When  a  major  system  acquisition  is  broken 
into  pieces  and  the  project  organization  acts  as  the 
“prime  contractor,”  there  is  a  clear  need  for  broad  and 
deep  system  engineering  to  make  sure  that  the  pieces 
can  go  together  again. 

Even  at  the  extreme  of  a  fixed-price  contract,  the 
acquisition  agency  must  have  sufficient  technical  cap¬ 
ability  to  monitor  the  contractor’s  progress,  understand 
the  nature  of  the  problems  uncovered,  and  where 
necessary,  assist  in  the  resolution  of  technical  problems. 
At  the  time  a  fixed-price  contract  is  negotiated,  the 
degree  of  definition  of  what  is  required,  the  specificity 
of  how  performance  will  be  measured,  and  the  con¬ 
fidence  in  achieving  the  desired  capability  within  cost 
and  schedule  goals  are  not  constant  and  are  affected 
by  a  number  of  factors.  The  degree  of  definition  at 
contract  time  is  affected  by  one's  ability  to  communi¬ 
cate  requirements  and  forecast  the  system  environment, 
the  need  for  statc-of-thc-art  development,  the  complex¬ 
ity  of  what  one  is  attempting  to  acquire,  the  amount  of 
prior  experience  with  the  type  of  system  being  acquired, 
and  the  process  by  which  one  arrived  at  the  contract 
stage. 

Equally  important  is  the  ability  to  define  at  contract 
time  the  manner  in  which  the  specified  performance 
once  acquired  will  be  managed  and  demonstrated.  Many 
of  the  same  factors  that  limit  the  accuracy  of  the 
specification  limit  one’s  ability  to  accurately  spell  out 
test  or  demonstration  measures. 

The  extent  to  which  the  factors  I  have  enumerated 
are  present  in  a  fixed-price  acquisition  determines  the 
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level  of  effort  required  to  provide  a  degree  of  confidence 
that  the  procurement  goals  (cost,  schedule,  and  per¬ 
formance)  will  be  met,  and  to  allow  the  acquisition 
agency  to  discharge  its  responsibilities  in  a  meaningful 
and  timely  manner. 

In  short,  and  under  any  circumstances,  periodic  re¬ 
views  of  the  design  and  production  activities  arc  neces¬ 
sary  if  unfortunate  surprises  are  to  be  avoided. 

Procurement  aspects 

While  it  is  not  the  purpose  of  this  paper  to  explore 
all  the  procurement  problems  associated  with  military 
real-time  data  processing  systems,  certain  of  these 
problems  and  related  considerations  are  of  such  direct 
interest  to  the  system  engineering  activity  that  they  will 
be  touched  on  here. 

Overall  arrangements 

In  a  system  program,  there  is  wide  variety  possible 
in  the  types  and  responsibilities  of  contractors,  methods 
of  selection  of  contractors,  and  types  of  contract.  There 
has  been  a  wide  variety  of  good  and  bad  experience 
with  these  different  alternatives,  and  no  single  alterna¬ 
tive  is  the  optimum  or  best  answer  in  all  circumstances. 
Any  can  work,  and  any  can  fail.  One  piece  of  guidance, 
however,  does  seem  clear:  the  acquisition  agency  can¬ 
not  and  must  not  relax  its  vigil  or  diminish  its  technical 
interest,  regardless  of  the  alternative  selected.  The  re¬ 
quirement  exists,  under  any  method  chosen,  for  a  strong 
competent,  and  responsive  technical  and  managerial 
capability. 

A  key  decision,  then,  will  be  that  of  how  to  augment 
the  project  organization’s  “in-house”  system  engineer¬ 
ing  capability.  The  use  of  independent  technical  organ¬ 
izations  furnishing  such  service  has  been  noted,  and 
their  use  is  sometimes  also  termed  “in-house.” 

There  are  two  major  approaches  to  acquisition  which 
help  limit  (but  not  eliminate)  the  need  for  in-house 
system  engineering  support  on  a  fixed-price  contract. 
These  approaches  help  to  pin  down  the  contract  re¬ 
quirements  by  providing  greater  insight  into  the  design 
method  or  by  providing  a  basis  for  hard  performance 
measures. 

By  use  of  the  “contract  definition”  approach,  in¬ 
volving  a  funded  definition  phase  with  two  or  more 
contractors  prior  to  final  selection,  the  majority  of  in- 
house  engineering  effort  can  be  applied  before  contract 
rather  that  after  contract.  The  results  of  the  effort  can 
be  a  set  of  design  specifications  rather  than  a  set  of 
performance  specifications.  Because  the  contractors 
arc  being  paid  for  their  design  effort  before  acquisition 
begins,  they  are  generally  more  responsive  in  providing 
design  details.  This  is  not  always  the  case  when  one  goes 
out  for  acquisition  with  performance  specifications  and 


tries  to  get  design  specifications  as  a  product  of  contract 
negotiation  effort.  Design  details  are  necessary  in  most 
systems  (when  performance  is  difficult  to  demonstrate) 
to  confidently  write  the  fixed-price  contract.  In  this 
regard,  I  would  like  to  point  out  that  performance  in¬ 
centives  may  do  more  harm  than  good  as  a  means  of 
satisfying  the  acquirer’s  needs  if  one  cannot  adequately 
(clear  and  unambiguously)  pre-specify  the  way  in 
which  performance  will  be  tested  and  demonstrated. 

A  second  approach  to  more  adequate  definition  in 
some  situations  is  the  use  of  prototypes,  or  the  “fly- 
before-buy”  approach.  If  one  constructs  and  tests  a 
sufficiently  comprehensive  prototype  of  a  system  be¬ 
fore  acquisition,  the  risks  in  acquisition  may  be  reduced. 

Even  though  both  of  these  approaches  ordinarily 
represent  significant  investments  in  time  and  dollars, 
they  do  not  guarantee  that  the  need  for  engineering 
support  besides  that  provided  by  the  contractor  will  be 
eliminated.  If  the  system  under  acquisition  is  pushing 
the  state-of-the-art,  if  a  complex  approach  is  being  used, 
if  the  performance  is  difficult  to  measure,  or  if  the 
number  of  interfaces  or  amounts  of  government-fur¬ 
nished  equipment  are  large,  the  need  for  extensive  in- 
house  engineering  effort  will  still  exist. 

Under  any  contracting  situation,  experience  points 
out  the  desirability  of  a  separate,  independent,  test  con¬ 
tractor,  who  is  not  responsible  for  the  original  design 
or  any  of  its  implementation,  to  help  assure  that  the 
system  is  properly  delivered,  installed,  cheeked  out, 
and  evaluated.  This  is  especially  true  in  the  ease  of 
multisite  systems. 

Software 

The  software  or  computer  programs  generally  repre¬ 
sent  the  most  costly  development  item  in  the  system 
acquisition.  As  such,  their  procurement  represents  a 
major  undertaking.  Computer  programs  can  be  acquired 
in  several  different  ways.  For  example,  software  can 
be  procured  from  equipment  manufacturers.  Software 
manufacturers  not  involved  in  equipment  production 
are  also  becoming  increasingly  available,  but  the  use 
of  a  company  different  from  that  which  supplies  the 
system  computer  can  introduce  some  difficulties  relating 
to  release  of  proprietary  information. 

Based  on  the  operational  job  to  be  done,  a  proper 
match  of  hardware  and  software  must  be  defined,  and 
neither  should  be  procured  without  first  considering  the 
implication  on  the  other.  In  cases  where  the  hardware 
and  software  represent  separate  procurements,  the 
hardware  is  normally  selected  first. 

Software  is  difficult  to  procure  under  fixed-price 
contracts  because  of  the  difficulty  of  producing  detailed 
performance  specifications  for  these  programs.  As  op¬ 
posed  to  hardware,  software  specifications  arc  most 
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often  in  the  language  of  the  user  rather  than  the  sup¬ 
plier.  As  with  hardware,  cost  should  not  be  the  primary 
or  only  selection  criterion  for  the  software  contractor. 
Experience,  past  performance,  geographical  location, 
and  availability  of  trained  manpower  should  weigh 
heavily  in  the  ultimate  selection. 

For  various  reasons,  on  several  occasions  more  than 
one  contractor  has  been  involved  in  the  software  prepar¬ 
ation.  As  noted  in  Section  VI,  there  are  several  major 
pieces  to  the  software;  this  has  led  to  a  false  conclusion 
that  these  pieces  are  completely  independent  and  can 
be  treated  as  such.  In  most  cases,  these  split  software 
contracts  have  led  to  unfortunate  and  unpleasant  dis¬ 
agreements,  regardless  of  whether  there  existed  a  sub¬ 
contractor  or  associate  contractor  relationship.  The 
answer  seems  to  be  to  avoid  split  software  arrangements 
if  possible.  If  not: 

•  A  single  agency  should  be  assigned  overall  de¬ 
sign  responsibility. 

•  Additional  resources  should  be  provided  to  mon¬ 
itor  the  separate  software  efforts. 

•  A  detailed  specification  of  the  program  interface 
should  be  prepared  and  used  as  a  basis  for  en¬ 
suring  program  compatibility. 

•  Special  care  should  be  excerciscd  in  establishing 
contractual  coverage  for  the  separate  software 
efforts  to  ensure  adherence  to  interface  specifica¬ 
tions. 

•  Special  provision  should  be  made  for  the  testing 
of  the  integrated  software. 

•  Satisfactory  working  relationships  between  the 
contractors  and  the  project  organization  should 
be  provided. 

As  noted  in  Section  Vi,  some  of  the  necessary  soft¬ 
ware  is  directly  associated  with  the  computer  and  com¬ 
prises  the  essential  general-purpose  support  programs 
with  which  to  produce  the  operational  and  other  sys¬ 
tem-specific  programs.  These  programs  are  customarily 
supplied  by  the  equipment  manufacturer,  but  there  is 
little  or  no  standardization  in  what  constitutes  this  basic 
support  software.  Hence  it  is  not  surprising  that  there 
have  been  a  number  of  misunderstandings  and  subse¬ 
quent  disappointments  concerning  what  the  computer 
manufacturer  was  to  deliver  in  this  category.  The 
situation  may  be  worse  if  the  programs  are  procured 
from  some  other  source.  Accordingly,  it  now  seems 
best  to  consider  the  basic  support  software  as  a  de¬ 
liverable  part  of  the  computer  hardware  procurement. 
Note  that  scheduling  of  the  delivery  of  the  support 
software  may  be  critical  since  it  is  required  for  use  in 
the  preparation  of  the  system  software. 

Software  procurement  should  be  made  on  the  basis 
of  a  plan  which  considers  the  longer  term  software 


problems.  A  major  consideration  is  whether  the  mili¬ 
tary  itself  will  take  over  software  production  or  main¬ 
tenance  at  some  point  in  time.  If  so,  early  introduction 
of  these  people  for  training  purposes  is  indicated. 

The  problem  of  correctly  estimating  the  software 
effort  is  reviewed  in  Section  VI.  A  related  problem  is 
that  of  measuring  actual  progress  during  the  software 
design  and  production  process  itself.  A  count  of  the 
actual  instructions  written  provides  only  a  limited 
measure,  in  view  of  the  initial  uncertainty  in  the  num¬ 
ber  required.  Further,  the  actual  preparation  of  the 
written  set  of  instructions  is  only  part  of  the  job,  with 
far  greater  uncertainties  lying  ahead  in  the  various 
phases  of  checkout.  The  result,  then,  is  that  progress 
in  the  early  phases  of  the  software  production  generally 
seems  to  go  according  to  schedule.  At  about  mid-point, 
delays  start  to  be  projected  and  slippages  begin  to  take 
place,  generally  on  at  least  a  one-to-one  basis  (i.e.,  a 
month  goes  by  and  the  delivery  date  has  slipped  by 
at  least  a  month).  Finally,  a  significant  slip  is  intro¬ 
duced  and  the  end  date  is  ultimately  reached. 

Inability  to  predict  end  dates  accurately  must  then 
be  expected  (and  hence  planned  for);  the  only  re¬ 
liable  prediction  will  be  one  made  just  before  end  date 
is  reached.  The  answer  from  the  procurement  point  of 
view  is  to  require  in  the  contractual  documents  that 
every  attempt  be  made  to  delineate  significant  mile¬ 
stones  than  can  be  measured,  and  to  insist  upon  and 
provide  close  monitoring  of  the  progress.  This  is,  un¬ 
fortunately,  easier  said  than  done,  and  much  improve¬ 
ment  of  procedure  and  techniques  is  required  here. 

Hardware 

In  view  of  the  significant  efforts  and  experience  in 
the  development  of  digital  computers  and  related  hard¬ 
ware  for  the  commercial  markets,  and  the  high  degree 
of  applicability  of  these  equipments  to  command  and 
control  applications,  there  has  naturally  been  great 
stress  placed  on  the  use  of  “off-the-shelf”  items.  Un¬ 
fortunately,  this  term  is  not  precise,  and  its  interpre¬ 
tation  has  ranged  from  “available  commercial  equipment 
in  production,”  to  “equipment  in  development,”  to 
“prototypes,”  to  “new  computer  designs  to  be  imple¬ 
mented  with  available  modules,”  and  even  to  mere 
ideas  (or  hopes).  There  is  great  difficulty  in  trying  to 
separate  fact  from  fancy. 

Because  of  the  long  lead  time  of  the  computer  pro¬ 
grams,  an  early  delivery  of  the  computer  is  generally 
essential  to  their  timely  checkout.  Thus  it  is  necessary 
that  great  caution  be  exercised  in  the  procurement  of 
the  computer,  to  ensure  its  “off-the-shelf”  availability. 
Accordingly,  an  early  (less  than,  say,  120  days)  avail¬ 
ability  of  a  computer  with  the  same  characteristics  as 
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the  ultimate  production  models  might  be  an  adequate 
proof  of  its  being  “off-the-shelf.” 

Generally,  each  system  has  the  need  for  some  rather 
unique  or  special  terminal  or  display  equipments,  so 
some  amount  of  development  is  usually  required.  The 
experience  here  has  been  very  bad,  both  with  delivery 
and  with  reliability.  Needless  to  say,  then,  hardware 
development  should  be  undertaken  only  when  abso¬ 
lutely  neeessary.  This  is  a  real  problem  for  displays, 
where  the  user  may  want  complex  multicolor  deviees 
that  are  not  truly  neeessary  for  successful  system  opera¬ 
tion.  This  adviee  is  easier  to  say  than  to  follow,  for 
there  is  always  pressure  for  gadgets  and  items  that 
are  “better  than  anybody  else’s.”  If  the  user  persists, 
the  acquisition  agency  must  make  the  projected  eost 
in  time  and  money  elear. 

Insofar  as  is  possible,  then,  scheduling  should  hon¬ 
estly  refleet  the  differences  between  development  items 
and  truly  “off-the-shelf”  equipment.  Where  possible, 
the  development  eyele  should  be  disassociated  from  the 
mainstream  system  acquisition.  Plans  should  attempt 
to  use  available  deviees  until  the  development  items 
are  ready. 

It  would  seem  there  should  be  monetary  rewards  and 
penalties  to  provide  additional  incentives  for  better 
contractor  schedule  performance.  Incentive  features 
are  certainly  no  panacea,  but  they  might  be  effective 
and  provide  more  usable  legal  leverage  than  the  threat 
of  termination  for  default. 

Finally,  caution  should  be  exercised  with  hardware 
procurements  based  on  detailed  design  specifications 
unless  the  procuring  agency  has  assured  itself  that  a 
produet  which  meets  these  specifications  will  provide 
the  desired  performance.  This  requires  significant  in- 
house  teehnieal  effort. 

///.  Broad  or  conceptual  design 

This  section  discusses  some  of  the  initial  considera¬ 
tions  whieh  should  guide  the  overall  design.  Detailed 
aspects  of  hardware,  software,  and  testware  design 
follow  in  succeeding  sections. 

What  is  the  system? 

Automated  eommand  and  eontrol  has  become  ex¬ 
tremely  fashionable  in  reeent  years,  achieving  some 
appeal  as  a  military  status  symbol.  This,  among  other 
reasons,  ean  lead  to  the  initiation  of  a  system  develop¬ 
ment  without  a  full  understanding  of  what  the  system 
is  and  what  it  is  expected  to  do.  When  this  happens,  the 
designer,  the  user,  and  the  taxpayer  may  later  regret 
the  initial  preeipitate  action. 

Too  often  we  hear  the  designer  lament  the  fact  that 
the  user  cannot  supply  him  with  an  adequate  statement 


of  requirements.  When  this  is  said,  it  usually  means 
that  the  designer  doesn’t  understand  the  nature  of  these 
systems  or  the  military  problem.  In  practice,  the  user’s 
requirements  are  not  easily  expressed  in  quantitative 
terms,  and  they  ean  only  be  put  into  meaningful  form 
for  procurement  when  matehed  with  the  available  tech¬ 
nology  and  possible  designs.  In  many  cases,  the  avail¬ 
able  technology  identifies  or  even  generates  the  require¬ 
ments. 

Thus,  as  the  very  first  step  in  the  design  effort,  the 
designer  and  user  must  eome  to  an  understanding  of  the 
military  problem,  and  they  should  prepare  and  docu¬ 
ment  a  matched  statement  of  requirements  and  the 
corresponding  conceptual  or  broad  design.  In  doing  so, 
the  full  implications  of  the  system  and  its  operational 
use  should  be  explored.  Both  parties  should  consider 
the  automated  portion  of  the  system  as  an  ieeberg,  and 
a  significant  initial  effort  should  be  directed  towards 
understanding  and  describing  what  is  under  the  surface. 

For  example,  the  implications  for  manning,  training, 
maintenance,  and  logisties  must  be  fully  understood 
by  the  user.  Too  often  we  are  disappointed  by  initial 
claims  of  savings  of  operational  personnel.  I  know  of 
no  system  in  whieh  aetual  personnel  savings  have  re¬ 
sulted,  although  in  most  cases  the  capability  (or  pro¬ 
ductivity)  of  the  operations  personnel  has  been  greatly 
increased. 

A  related  question  requiring  an  early  answer  is: 
“What  constitutes  the  system?”  In  particular,  what 
equipments  already  in  use  by  the  manual  system  are 
to  be  employed?  In  determining  this,  the  operating 
condition  of  these  field  equipments  should  be  realistical¬ 
ly  ascertained.  The  early  development  of  SAGE  was 
hampered  by  the  fact  that  the  radars  were  not  con¬ 
sidered  as  a  part  of  the  system.  In  both  SAGE  and 
412L,  the  aetual  operating  condition  of  sueh  field 
equipments  was  not  properly  understood,  and  diffi¬ 
culties  resulted. 

A  eareful  inventory  and  field  survey  is  reeommended. 
This  applies  to  both  equipments  and  facilities.  The 
system  designer,  often  an  electronies  engineer,  is  quite 
prone  to  ignore  the  seemingly  prosaic  problems  or 
roads,  buildings,  power,  and  air  conditioning.  Measure¬ 
ments  on  field  equipments  under  field  maintenance  are 
advisable.  A  radar  under  field  conditions  performs 
quite  differently  than  at  the  manufacturer’s  plant.  SAGE 
and  412L  experience  attest  to  this. 

At  this  early  phase,  the  interfaces  with  other  sys¬ 
tems — present  and  future — must  be  considered.  An 
exchange  of  data  with  other  systems  is  almost  always 
required,  and  the  problems  of  data  compatibility — in 
terms  of  information  content,  format,  and  signal  levels — 
must  be  solved.  Not  only  should  these  problems  be 
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considered  at  the  design  level,  but  responsibility  for 
any  changes  to  achieve  compatibility  must  be  clearly 
assigned. 

Briefly  then,  at  the  outset  there  should  be  a  careful 
consideration  and  documentation  of  what  the  system  is, 
what  it  will  do,  what  comprises  it,  and  how  it  relates 
to  other  systems. 

Evolution 

As  noted  earlier,  significant  periods  of  time  are  in¬ 
volved  in  the  acquisition  of  these  systems.  When  they 
do  become  available,  they  may  not  meet  the  then  cur¬ 
rent  requirements  and  may  be  very  difficult — in  terms 
of  time  or  effort — to  modify  so  as  to  achieve  the  desired 
goals.  This  has  given  rise  in  recent  years  to  a  general 
feeling  that  the  design  and  implementation  of  these 
systems  should  be  pursued  on  a  more  evolutionary  basis. 

The  merits  of  an  evolutionary  approach  are  said  to 
be  that  we  can  proceed  in  small,  sure  steps.  It  would 
provide  for  an  early  demonstration  of  capability  and 
would  ensure  operational  experience  and  feedback  be¬ 
fore  proceeding  with  a  more  sophisticated  design.  Head¬ 
quarters  command  systems  may  be  strongly  influenced 
by  the  personality  and  desires  of  the  commander 
himself,  and  an  evolutionary  capability  would  make  it 
possible  for  individual  commanders  to  tailor  or  modify 
the  system  accordingly.  It  is  further  felt  that  the  sophis¬ 
tication  and  technical  difficulties  which  have  character¬ 
ized  the  first  generation  of  these  systems  could  be 
avoided  in  the  future  if  there  were  greater  use  of  “off- 
the-shelf’  equipment  and  a  more  direct  initial  mechani¬ 
zation  of  existing  manual  operations. 

The  current  emphasis  on  evolution,  however,  may 
eonfuse  what  are  perhaps  two  separate  ideas.  The  first 
is  the  concept  of  a  time-phased  implementation,  or 
what  might  be  termed  a  planned  evolution  towards  a 
predetermined  design.  The  sceond  relates  to  provisions 
for  the  unplanned  modifications  necessitated  as  a  re¬ 
sult  of  operating  experience  or  changes  in  requirements 
or  the  technology. 

In  this  light,  the  evolutionary  approach  has  some 
dangers  which  should  not  go  unrecognized.  In  par¬ 
ticular,  a  time-phased  implementation  should  not  be 
used  as  an  excuse  for  avoiding  the  total  system  design 
problem.  If  only  the  first  increment  is  planned,  then 
there  is  a  strong  possibility  that  inadequate  attention 
will  be  given  to  the  design  requirements  imposed  by 
subsequent  steps.  Specifically,  the  universal  lack  of 
funds  and  the  influence  of  ever-present  economy  drives 
eause  strong  pressure  in  this  direction  and  may  force 
a  decision  to  buy  only  the  equipment  or  capability  re¬ 
quired  for  the  first  steps  and  not  allow  time  for  suf¬ 
ficient  analysis  and  design  of  the  total,  long-range 


system.  Restrictions  on  building  size,  power,  or  air 
conditioning  ean  be  as  serious  as  limited  computer 
capability.  Subsequent  steps  are  then  very  difficult 
and  costly  to  implement,  possibly  requiring  grossly 
different  equipments  and  eostly  retrofits  to  existing 
equipments. 

Another  danger  of  time-phased  implementation  is 
that  if  the  subsequent  steps  are  not  very  earefully 
planned,  major  problems  ean  arise  as  these  steps — in¬ 
volving  additional  programs  and  possibly  equipments — 
introduce  interference  with  the  operation  of  the  sys¬ 
tem.  A  related  difficulty  is  the  retraining  of  operators. 
In  faet,  there  appears  to  be  a  critical  mechanization 
level  whieh  should  be  incorporated  in  an  initial  step. 
This  should  include  the  full  mechanization  of  the  es¬ 
sential  inputs  and  outputs,  and  should  include  the 
majority  of  the  operator  facilities  (consoles,  keyboards, 
switches,  cte.),  even  though  all  of  this  capability  might 
not  be  used  initially. 

It  should  further  be  noted  that  requirements  for  test¬ 
ing  are  such  that  initial  system  testing  is  not  in  any 
large  percentage  salvageable  for  subsequent  steps,  and 
a  complete  scries  of  tests  with  all  attendant  costs  may 
be  required  for  the  replacement  or  next  model  of  the 
system. 

Another  point  is  that  while  direet  mechanization  of 
existing  procedures  may  yield  an  early  demonstration 
of  progress,  it  should  be  applied  with  eaution,  as  it 
may  prevent  the  realization  of  a  vastly  improved  op¬ 
eration  which  eould  be  achieved  by  fully  exploiting  the 
automatic  data  processing  capability.  (Neither  the 
automobile  nor  the  airplane  were  products  of  direct 
mechanization. ) 

There  is  also  the  question  of  the  use  of  “off-the- 
shelf”  equipments.  This,  too,  must  be  approached  with 
eaution  and  reason.  Beyond  the  computers,  there  is 
very  little  that  is  actually  “on-the-shelf,”  particularly 
in  the  sense  that  it  ean  be  applied  direetly  to  a  system. 
As  noted  earlier,  it  is  extremely  important  to  differen¬ 
tiate  between  “off-the-shelf”  and  “in  the  broehure.” 

Design  guidance  in  this  broad  area,  then,  would  seem 
to  be  as  follows: 

•  The  design  should  contain  a  reasonable  degree 
of  exeess  eapaeity  and  flexibility  in  all  subsys¬ 
tems  and  elements  to  permit  long-term  unplanned 
evolution.  This  exeess  capacity  must  then  be 
earefully  controlled  so  that  it  does  not  dwindle 
away  before  it  is  needed.  It  should  not  be  used 
for  “frills”  which  prevent  its  future  use  for 
“needs.” 

•  If  the  system  is  to  be  implemented  in  steps,  the 
design  must  eonsider  the  resulting  system  first, 
allowing  sufficient  eapaeity  to  achieve  this  goal. 
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Only  when  this  end  design  is  well  understood 
should  the  steps  in  the  implementation  be  de¬ 
lineated. 

•  The  first  step  should  not  be  too  big  to  prevent 
an  early  demonstration  of  progress,  but  should 
be  sufficiently  large  to  include  those  features 
strongly  affecting  operator  actions  and  activities 
as  well  as  those  features  relating  to  integration 
with  other  systems. 

•  The  design  of  a  step-wise  implementation  must 
not  ignore  the  problems  of  disruption  and  deg¬ 
radation  of  current  operations. 

•  The  designer  should  consider  with  caution  the 
early  benefits  of  a  direct  mechanization  and  the 
applicability  and  availability  of  “off-the-shelf'’ 
equipments. 

Degree  of  automaticity 

The  proper  degree  of  automation  to  be  provided  in 
most  command  and  control  systems  is  not  easily  de¬ 
termined.  Many  system  functions  are  routine,  easily 
described,  repetitive  data  processing  tasks  which  lend 
themselves  directly  to  mechanization;  other  functions 
clearly  require  sophisticated  choices  or  decisions  which 
can  or  should  involve  human  judgment.  Unfortunately, 
however,  a  larger  number  of  important  functions 
usually  cannot  be  so  easily  categorized.  Here  the  cost 
and  complexities  of  automation  must  be  balanced  against 
the  problems  and  difficulties  attendant  of  human  op¬ 
erators. 

First,  the  man-machine  relationship  is  severely  limit¬ 
ed  by  the  cost  and  complexity  of  the  devices — consoles 
or  large-screen  displays — available  for  the  information 
exchange.  The  operator  can  do  a  satisfactory  job  only 
if  he  is  given  the  necessary  data  upon  which  to  make 
his  decision.  This  is  surprisingly  difficult  and  costly  to 
accomplish,  both  in  terms  of  hardware  and  software. 

Second,  the  involvement  of  an  operator  introduces 
both  equipment  and  human  reaction  times.  Delays  can 
arise  both  in  the  computer  program  and  in  the  display 
devices.  Consoles  may  introduce  several  seconds  of 
delay,  depending  on  the  storage  medium  which  drives 
them;  large  screen  displays  may  involve  processing  and 
mechanical  transport  delays  of  ten  seconds  or  greater; 
keyboards  provide  a  flexible  input  capability,  but  re¬ 
quire  significant  times  for  message  composition. 

Third,  the  operator  may  not  have  the  information  re¬ 
quired  to  make  a  better  decision  than  the  computer. 
In  SAGE,  it  was  felt  that  whenever  the  automatic 
tracking  program  got  into  difficulty  as  a  result  of  too 
little,  too  much,  or  ambiguous  data,  the  operator  would 
be  alerted  to  correct  the  situation.  An  evaluation  of  one 
automatic  track  monitoring  scheme  later  showed  that 


the  operators  did  not,  on  the  average,  improve  the 
situation  over  what  the  computer  would  have  done  if  it 
had  been  left  alone. 

The  other  side  of  tradeoff  is  that  the  automation  of 
complex  decisions  can  be  extremely  difficult  and  costly 
in  computer  capacity.  A  satisfactory  automatic  weapons 
assignment  program  must  consider  a  multitude  of 
factors:  position,  heading,  and  altitude  of  bomber  air¬ 
craft;  position  and  importance  of  possible  targets;  lo¬ 
cations,  status,  types,  and  capabilities  of  available 
weapons;  and  so  forth.  Other  decisions  are  not  as  simple 
to  automate  as  they  first  appear.  Early  in  the  system 
design  there  is  a  disturbing  tendency  to  oversimplify 
and  underestimate,  and  we  are  subsequently  embar¬ 
rassed  by  the  computer  time  or  storage  consumed  by 
what  had  been  felt  to  be  a  simple  process. 

Let  me  mention  two  examples  from  SAGE.  The  first 
is  automatic  initiation  of  tracks  from  radar  replies  re¬ 
ported  by  a  scanning  radar.  This  was  first  said  to  be  a 
simple  pattern  recognition  process:  look  for  radar 
replies  in  close  proximity  on  successive  scans,  allowing 
some  leeway  for  tracks  with  low  blip-scan  ratios. 
In  actual  practice,  a  high  level  of  noise  replies  requires  a 
very  sophisticated  process  if  the  wholesale  generation 
of  false  tracks  is  to  be  avoided  or  if  the  initiation  of  real 
tracks  is  not  to  be  overly  delayed. 

Track-while-scan  of  aircraft  is  still  dismissed  by 
many  people  as  a  simple  process.  Even  if  they  include 
such  considerations  as  low  blip-scan  ratios,  false  tracks, 
and  turning  tracks,  they  generally  have  ignored  the 
problems  of  proximate  or  crossing  tracks.  This  involves 
another  level  of  design  sophistication. 

In  both  cases,  it  wasn’t  until  a  rather  detailed  knowl¬ 
edge  of  the  actual  physical  situation  became  available 
and  extensive  computer  programming  had  been  per¬ 
formed  that  a  full  realization  of  the  cost  and  complexity 
was  reached. 

Finally,  decisions  as  to  the  degree  of  automation 
must  be  carefully  made  in  light  of  operator  limitations 
and  training  problems.  The  danger  of  the  engineer  de¬ 
signing  the  system  for  himself  to  operate  is  ever  present. 
The  performance  of  many  of  our  automated  systems — 
SAGE  and  412L  are  examples — is  overly  sensitive  to 
operator  qualifications  and  training.  This,  in  turn, 
places  a  very  high  premium  on  provisions  for  continuous 
exercising  and  evaluation  of  these  operators.  Such  exer¬ 
cising  and  evaluation,  as  is  discussed  next,  is  not 
without  its  own  costs  and  problems.  An  even  better 
solution,  where  possible,  is  to  provide  the  system  with 
a  meaningful  day-to-day  job.  The  integration  of  air 
defense  and  air  traffic  control  has  always  seemed  to  be 
a  natural  combination.  It  is  a  pity  that  political  and 
other  constraints  seem  to  prevent  this  union. 
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Self-exercising  capability 

As  previously  noted,  during  peacetime  these  systems 
normally  operate  at  rather  low  load  levels,  far  below 
design  capacity.  Since  they  arc  quite  sensitive  to  op¬ 
erator  performance,  the  best  possible  system  perform¬ 
ance  will  be  attained  when  required  only  if  a  self- 
exercising  capability  to  maintain  and  improve  operator 
proficiency  has  been  included  as  an  integral  part  of 
the  overall  design.  Such  a  capability,  by  which  the 
system  can  be  subjected  to  simulated  high  Loads  and 
special  input  conditions,  will  also  permit  on-going 
evaluation,  checks  on  system  performance,  and  those 
demonstrations  of  system  performance  which  are  often 
required  for  visitors. 

As  a  part  of  the  system  design,  then,  careful  attention 
should  be  given  to  the  questions  of  which  environmental 
conditions  and  sources  of  input  data  should  be  simulated 
and  how  this  can  best  be  done.  Additional  equipments 
are  generally  required  for  this  purpose. 

If  the  system  has  many  sources  of  data,  the  generation 
of  consistent  inputs  relating  to  the  same  environmental 
situation  may  in  itself  constitute  a  difficult  task  requiring 
data  processing  equipment.  For  example,  in  an  air 
defense  system  it  is  desirable  to  simulate  enemy  bomber 
flights  as  seen  by  a  radar.  It  is  possible  to  do  this  in 
real  time  by  suitable  analog  or  digital  simulators  at  a 
radar  site,  or  the  simulated  radar  returns  can  be  pre¬ 
pared  and  prerecorded  on  a  magnetic  tape  or  photo¬ 
graphic  film  which  can  be  scanned  by  appropriate 
equipment  at  the  site.  Either  of  these  techniques  would 
be  suitable  for  simulation  at  an  individual  site.  However, 
if  the  system  consists  of  many  radars  with  overlapping 
coverage,  and  the  aircraft  will  traverse  the  coverage 
of  several  sites,  it  becomes  necessary  to  simulate  a 
consistent  set  of  radar  inputs.  This  generally  requires 
the  coordinated  generation  of  tape  or  film  inputs.  This, 
in  turn,  entails  a  significant  data  processing  task  in  the 
preparation  of  aircraft  flight  paths  and  the  computa¬ 
tion  of  the  r,  0  data  as  observed  by  the  radars.  A 
general-purpose  computer  may  be  useful  or  necessary 
for  this  purpose. 

In  any  event,  is  is  generally  advantageous  for  train¬ 
ing  purposes  to  arrange  the  simulation  so  that  it  is 
repeatable  and  situations  can  be  easily  reproduced  for 
the  same  or  different  sets  of  operators. 

A  second  problem  of  exercising  is  that  of  concurrent 
operation  with  live  inputs.  Even  if  it  is  electronically 
possible  to  mix  the  live  and  simulated  inputs,  this  may 
cause  extreme  confusion  and  danger  unless  the  computer 
has  some  means  of  segregating  and  identifying  each. 
This  may  require  excess  computer  capacity  for  the 
extra  processing  and  display  programs  required  to 
differentiate  between  the  simulated  and  live  inputs,  at 


least  to  selected  operating  personnel. 

Some  simulated  inputs  cannot  be  preplanned  since 
they  are  affected  by  the  system  operation  and  may  re¬ 
quire  real-time  data  processing  and  human  decisions. 
During  a  SAGE  training  exercise  in  which  a  center  is 
removed  from  the  air  defense  net,  voice  radio  vectoring 
instructions  are  monitored  by  simulated  pilots  who 
insert  the  directed  values  of  heading,  speed,  and  altitude 
into  the  computer  by  special  switches.  A  special  com¬ 
puter  program  uses  these  inputs  to  simulate  the  flight 
paths  of  the  interceptors.  The  current  positions  of  these 
aircraft  are  then  determined,  permitting  the  computation 
of  the  simulated  radar  returns.  Thus,  system  exercising 
involving  non-preplanncd,  dynamic  inputs  may  require 
special  input  facilities,  added  operating  personnel,  and 
additional  data  processing  capability. 

Finally,  adequate  exercising  facilities  must  also  pro¬ 
vide  the  means  for  evaluating  the  system  performance. 
Rapid  feedback  to  operational  personnel  is  a  require¬ 
ment  if  they  are  to  understand  and  correct  errors.  De¬ 
pending  on  the  situation,  real-time  evaluation  or  post¬ 
test  analysis  of  recorded  data  can  be  employed.  This 
evaluation  and  analysis  is  not  without  cost,  facilities, 
and  design  effort. 

It  should  be  noted  that  the  facilities  required  for 
system  exercising  are  closely  related  to  those  required 
for  program  and  system  checkout,  shakedown,  and  test 
The  facilities  should  be  coordinated  and,  where  possible, 
combined. 

Performance  monitoring 

As  systems  become  larger  and  more  complex  and 
particularly  as  the  number  of  input  and  output  sub¬ 
systems  grow,  it  becomes  increasingly  difficult  to  de¬ 
termine  whether  the  overall  system  is  operating  properly 
and  to  isolate  the  causes  of  difficulty.  Accordingly, 
suitable  provisions  for  performance  monitoring,  trouble 
detection,  and  quality  control  should  be  included  as 
part  of  the  system  design.  The  requirement  for  con¬ 
tinuous  system  operation,  coupled  with  the  relatively 
abundant  opportunities  for  subsystem  malfunction, 
means  that  the  designer  should  not  expect  that  per¬ 
formance  checks  during  preventive  maintenance  periods 
will  suffice.  He  must  consider  the  need  for  dynamic  or 
real-time  performance  monitoring  and  diagnosis  of 
malfunctions. 

The  central  computer  and  the  computer  program  are 
not  major  problems  in  this  regard.  The  majority 
of  their  random  or  intermittent  malfunctions  can  be 
detected  by  parity  checking,  and  in  many  cases  an 
immediate  repeat  of  the  computer  or  program  operation 
can  be  successfully  conducted.  Other  errors  generally 
cause  an  obvious  system  malfunction  or  complete 
stoppage.  However,  the  input  and  output  subsystems 
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offer  many  opportunities  for  misealibration,  random 
errors,  poor  performance,  and  complete  outage  in  a 
fashion  that  docs  not  cause  the  system  to  cease 
operation.  Individual  communication  links,  particularly 
those  carrying  digital  data,  can  become  completely 
inoperative;  individual  radars  can  go  out  of  calibration 
or  can  fail  to  report  targets  entirely;  or  individual  con¬ 
soles  or  manual  input  devices  can  fail  entirely — all  for 
significant  periods  of  time  before  the  degradation  or 
failure  becomes  apparent  to  operating  personnel. 

Most  subsystems  have  some  “built-in”  performance 
monitoring:  parity  alarms,  power  lights,  etc.  However, 
in  many  instances  the  subsystem  equipments,  their 
failure  indicators,  and  those  who  maintain  these  equip¬ 
ments  arc  remotely  located,  far  from  the  eventual  users 
of  the  data  which  they  provide.  Experience  indicates 
that  attention  to  the  subsystem  performance  may  only 
occur  when  the  human  element  comes  into  play — 
specifically,  when  the  user  complains.  Hence  an  ob¬ 
jective  of  the  system  performance  monitoring  should 
be  to  provide  the  user  with  the  tools  and  information 
to  complain  accurately. 

The  designer  should  consider  the  automatic  and 
periodic  generation  of  test  messages  that  can  be  routed 
through  these  subsystems  in  a  systematic  manner  for 
cheeking  purposes.  When  difficulties  are  encountered, 
more  specific  and  detailed  check  messages  or  techniques 
can  be  called  upon  to  isolate  the  particular  source  of 
the  difficulty.  In  many  cases,  it  may  be  possible  to 
correct  or  bypass  the  source  of  the  difficulty  temporarily 
until  it  can  be  fixed.  Finally,  a  continuous  summary  of 
the  status  of  the  key  elements  of  the  system  should  be 
made  available  to  both  operating  and  maintenance  per¬ 
sonnel. 

Two  examples  from  SAGE  might  help  to  illustrate 
the  general  ideas.  In  the  first,  the  radar  data  processing 
devices  at  each  radar  site  were  designed  with  a  provision 
for  periodically  introducing  false  targets  at  predeter¬ 
mined  range  and  azimuth  positions,  and  the  central 
computer  isolates  and  processes  these  messages  to 
check  a  number  of  the  processing  and  communication 
facilities  between  the  radar  and  the  central  computer. 

The  second  example  relates  to  the  importance  in  the 
SAGE  design,  as  with  other  netted  radar  systems,  of 
accurate  alignment  of  the  radars  in  both  range  and 
azimuth.  If  they  are  misaligned,  the  generation  of 
multiple  data  trails  and  false  tracks  for  a  single  aircraft 
may  result.  Since  both  the  radar  and  radar  data  process¬ 
ing  equipments  can  become  misaligned  during  mainte¬ 
nance  on  the  antenna,  decoders,  etc.,  a  performance 
monitoring  feature  was  added  to  the  SAGE  computer 
program.  With  the  feature,  the  computer  cheeks  reports 
from  different  radars  on  the  same  aircraft  and  deter¬ 


mines  whether  a  better  match  of  incoming  data  would 
exist  if  bias  errors  are  assumed  in  the  azimuth  data. 
If  it  is  found  that  an  azimuth  correction  on  a  radar 
improves  the  consistency  of  multiple  radar  reports, 
this  error  value  is  then  introduced  before  processing 
subsequent  reports.  The  error  is  also  printed  out  for 
corrective  action  at  the  radar  site  at  the  next  main¬ 
tenance  period.  (Subsequent  experiences  indicate  that 
automatic  correction  may  be  desirable  only  in  critical 
situations;  this  feature  has  the  possible  negative  effect 
of  minimizing  the  pressure  on  correcting  the  source  of 
the  error.) 

Performance  monitoring,  then,  should  be  recognized 
as  a  system  function  on  a  par  with  the  more  familiar 
operational  functions.  It  should  not  be  allowed  to  slip 
into  the  background  during  the  system  design.  Utiliza¬ 
tion  should  be  made  of  the  performance  monitoring  built 
into  existing  equipment  subsystems,  but  added  hard¬ 
ware  and  software  may  be  required  and  this  should  be 
reflected  into  the  design  of  system  equipments,  com¬ 
puter  programs,  and  communication  links. 

Continuity  and  modes  of  operation 

It  will  be  desirable  to  subdivide  this  topic  into  two 
parts.  The  first  part  deals  with  continuity  of  automatic 
system  operation  under  normal  or  peacetime  conditions. 
The  second  part  relates  to  alternate  modes  of  operation, 
generally  at  lower  eapaeity  and  capabilities,  when  key 
elements  of  the  system  have  been  put  out  of  operation. 
For  ease  of  reference,  normal  operation  will  be  referred 
to  as  Mode  I,  an  alternate  automatic  arrangement  as 
Mode  II,  and  a  completely  manual  back-up  as  Mode 
III. 

Failures  of  critical  system  elements  will  cause  an 
interruption  of  Mode  I  operation  unless  adequate  design 
measures  have  been  taken.  The  usual  solution  is  the 
provision  of  duplicate  or  duplex  equipments  coupled 
with  error  detection  facilities  and  a  rapid  switchover 
capability.  In  the  case  of  one-of-a-kind  equipments, 
full  duplicates  would  be  required;  in  the  ease  of  a 
multiplicity  of  units — memory  units,  display  consoles, 
tape  drives,  etc. — only  a  few  spare  elements  might  be 
needed.  In  the  case  of  consoles,  a  general-purpose  con¬ 
sole  design  rather  than  special-purpose  consoles 
designed  specifically  for  an  operational  position 
should  be  considered.  With  suitable  program  parameter 
and  switch  label  changes,  it  is  possible  to  adapt  a 
general-purpose  console  to  individual  positions,  thereby 
retaining  desired  flexibility  and  permitting  rapid  re¬ 
placement  or  substitutions  in  event  of  console  failures. 

With  improvements  in  equipment  reliability,  pro¬ 
vision  of  duplicate  units,  and  suitable  design,  unsched¬ 
uled  interruptions  of  Mode  I  continuity  ean  be  brought 
down  to  whatever  low  level  is  desired.  The  designer 
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must  not  overlook,  however,  requirements  leading  to 
scheduled  interruptions  of  Mode  I  continuity.  These 
include  the  functions  of  maintenance,  equipment  retrofit, 
program  retrofit,  and  possibly  system  exercising. 

Maintenance  is  self-explanatory.  Equipment  and  pro¬ 
gram  retrofit  may  result  from  initial  design  short¬ 
comings  or  from  the  evolving  nature  of  the  system. 
Program  retrofit  must  include  a  thorough  checkout  on 
the  actual  machine,  and  is  not  as  simple  as  merely 
changing  a  tape  unit  and  reading  in  a  new  program.  As 
noted  earlier,  system  exercising  may  require  interrup¬ 
tion  of  normal  operation  unless  the  live  and  simulated 
inputs  are  properly  handled. 

It  is  important  not  to  underestimate  the  amount  of 
downtime  required  to  perform  these  functions.  This  is 
particularly  true  in  the  early  months  or  years  of  opera¬ 
tion  where  several  hours  may  be  required  each  day.  On 
a  long-term  basis,  daily  maintenance  or  exercising  may 
still  be  required,  with  only  occasional  changes  to  the 
hardware  or  software. 

A  spare  or  duplicate  unit  for  each  type  of  element 
in  the  system  may  permit  continuity  of  Mode  I  operation 
during  limited  types  of  equipment  maintenance  and 
retrofit.  A  full  duplex  would  permit  a  Mode  I  opera¬ 
tion  while  performing  any  of  the  functions.  Reverting  to 
Mode  II  or  Mode  III  operation  is  also  possible,  although 
the  latter  is  not  very  desirable. 

Duplexing  will  not,  of  course,  prevent  physical  de¬ 
struction  of  the  key  elements  of  the  system  and  the 
consequent  interruption  of  Mode  I  operation.  Beyond 
the  measures  of  hardened  construction  or  mobility, 
some  added  survivability  can  be  achieved  by  a  dispersed 
or  decentralized  design.  In  some  systems,  a  decentralized 
design  utilizing  several  data  processing  centers  may 
be  required  by  economic  or  other  considerations.  For 
example,  a  single  air  defense  center  might  be  technically 
feasible  for  all  of  the  United  States,  but  the  communi¬ 
cations  costs  from  outlying  radar  sites  would  be  quite 
expensive.  A  completely  decentralized  design,  with  a 
computer  center  at  each  radar  site,  may  be  equally 
expensive  due  to  the  cost  of  communications  among 
the  centers,  but  this  design  is  generally  more  survivable. 

When  several  centers  are  involved,  however,  it  be¬ 
comes  possible  to  arrange  for  a  Mode  II  operation  in 
which  one  or  more  centers  can  “cover”  for  an  adjacent 
one.  Excess  data  processing  capacity  and  the  necessary 
communication  links  must  be  provided. 

The  last  and  least  attractive  possibility  in  the  event 
of  Mode  I  failure  is  to  revert  to  a  completely  manual, 
Mode  III,  operation.  Needless  to  say,  the  capacity  and 
capability  are  not  attractive,  and  there  is  the  serious 
economic  problem  of  maintaining  and  exercising  the 
added  operational  personnel.  It  is  easily  seen  that  Modes 
I,  II,  and  III  cannot  be  considered  independently.  The 


proper  selection  and  design  of  these  modes  has  a  direct 
bearing  on  the  system  configuration  and  on  almost 
every  aspect  of  the  design. 

Overhead  facilities 

At  the  early  stages  of  system  design  and  planning, 
attention  should  be  given  to  the  following  non- 
operational  functions: 

•  Training  of  operational  and  maintenance  person¬ 
nel. 

•  Initial  and  on-going  program  production. 

•  On-going  test  and  evaluation  of  evolving  system 
changes  and  additions. 

•  Generation  of  exercise  materials. 

•  Reduction  and  analysis  of  data  recorded  during 
system  operation. 

These  system  support  functions — as  opposed  to 
those  support  functions  which  must  be  conducted  at 
each  site — generally  require  separate  “overhead”  facili¬ 
ties,  owing  to  the  large  computer  time  requirements  and 
special  conditions  involved. 

The  first  three  functions — personnel  training,  pro¬ 
gram  production,  and  test  and  evaluation — require 
system-like  equipments  in  system-like  configurations, 
although  perhaps  not  with  the  added  equipments 
(added  modules  or  full  duplexing)  required  for  very 
high  reliability  and  continuity  of  operation.  The  last 
two  functions — generation  of  exercise  materials  and 
data  reduction  and  analysis — can  use  systcm-like  equip¬ 
ments,  but  could  also  use  other  computers  or  commercial 
computing  facilities. 

Depending  upon  the  specific  nature  of  the  system, 
one  overhead  facility  might  be  sufficient  to  handle  the 
first  three  functions.  In  large  systems  this  may  not  be 
possible  because  of  the  requirements  on  computer 
time.  In  SAGE — with  about  22  operational  direction 
centers  at  peak — the  number  of  such  overhead  facilities 
has  decreased  from  a  high  of  five,  and  until  1964  at 
least  one  computer  was  devoted  to  each  of  these 
functions. 

An  attractive  possibility  is  to  use  the  overhead 
machine(s)  as  a  part  of  the  baek-up  or  Mode  II  con¬ 
figuration.  In  this  case  and  when  an  operational  machine 
is  unavailable,  an  overhead  machine  could  cease  its 
non-operational  functions  and  join  the  operational  net. 
This  might  be  considered  as  a  form  of  duplexing,  with 
the  two  machines  at  separate  locations. 

Overhead  facilities,  particularly  those  required  for 
training  and  program  production,  assume  added  im¬ 
portance  since  they  are  generally  required  prior  to  the 
operational  facilities.  An  overhead  facility  for  subse¬ 
quent  test  and  evaluation  can  be  used  early  in  the  life 
of  the  system  for  the  design  verification  described  in 
the  next  section. 
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IV .  Design  process  and  techniques 

Volumes  have  been  written  on  the  subject  of  the 
different  tools  and  techniques — ranging  from  probability 
theory  to  simulation — which  can  be  employed  by  the 
system  designer  or  engineer.  How  these  can  best  be 
used  and  how  the  design  effort  should  proceed  are 
much  more  difficult  to  reduce  to  writing.  Beyond 
stating  that  design  is  both  analysis  and  synthesis,  that 
it  must  involve  feedback,  and  that  it  is  hardly  ever 
finished,  I  intend  here  to  mention  only  two  key  tech¬ 
niques — design  documentation  and  design  verification — 
which  have  been  found  to  be  of  practical  utility  in 
many  systems. 

Design  documentation 

The  need  for  an  early  agreement  on  the  requirements 
and  a  matching  conceptual  design  has  been  noted.  The 
design  effort  starts  here  and  must  bring  into  play  full 
consideration  of  costs  and  schedules.  Much  interplay, 
much  give  and  take,  and  much  analysis  and  synthesis 
may  be  required.  The  product  should  be  an  operational 
plan,  an  employment  plan,  or  some  other  suitably 
named  document  which  will  serve  as  the  broad  plan  or 
prospectus  for  the  system. 

At  an  early  point  in  the  design  effort,  careful  thought 
should  be  given  to  the  types  and  levels  of  documenta¬ 
tion  to  be  used.  Documentation  is  one  of  the  key 
management  tools  for  the  design  effort,  with  regard  to 
both  the  timeliness  and  the  quality  of  its  execution,  as 
well  as  its  conformance  to  user  and  other  requirements. 
Documentation,  then,  is  a  design  technique;  it  is  a  key 
to  organizing  a  design  effort  and  to  maintaining  design 
control.  The  time  and  effort  devoted  to  generating 
and  maintaining  the  documentation  will  be  very  well 
spent. 

The  names  of  the  documents  are  not  important.  To 
quote  merely  a  few:  Operational  Specifications,  Mathe¬ 
matical  Specifications,  System  and  Subsystem  Per¬ 
formance  Specifications,  Functional  Specifications, 
System  and  Subsystem  Design  Specifications,  Computer 
Program  Specifications,  and  Equipment  Specifications. 
What  is  important  is  that  there  be  recognized  levels  of 
documentation  and  that  the  responsibilities  for  each  be 
assigned.  Too  often,  the  design  is  not  properly  docu¬ 
mented  and  hence  not  available  to  those  who  must 
be  brought  to  bear  on  the  production  and  implementa¬ 
tion  effort  when  the  system  has  been  broken  down  into 
the  smaller  pieces  required  for  such  activities. 

As  noted,  at  the  highest  level  there  should  be  an 
overall  description  of  what  the  system  will  do  and  how 
it  will  be  deployed.  Next,  the  overall  performance  of 
the  system — at  least  in  qualitative  terms — should  be 
described,  including  identification  and  definition  of  the 


principal  subsystems.  Such  system  performance  speci¬ 
fications  might  be  the  vehicle  for  contracting  with  a 
they  should  be  held  to  meeting  subsystem  performance 
prime  contractor.  If  associate  contractors  are  used, 
specifications.  In  most  cases,  the  detailed  design  speci¬ 
fication  describing  what  is  built  will  be  prepared  by  the 
contractors. 

In  this  regard,  it  should  be  noted  that  it  is  exceed¬ 
ingly  important  to  document  a  baseline  system  con¬ 
figuration  at  the  earliest  date.  This  configuration  has 
a  first-order  impact  on  facilities,  construction,  civil 
engineering,  and  the  government-furnished  equipments. 
The  documentation  of  this  configuration,  ultimately 
describing  all  facilities,  equipments,  computer  programs, 
and  technical  manuals  in  the  system,  must  be  ade¬ 
quately  maintained  and  all  changes  controlled. 

Design  verification 

The  second  design  technique  is  design  verification. 
By  this  is  meant  any  reasonable  steps  which  can  be 
taken  to  validate  the  adequacy  of  the  design  at  the 
earliest  possible  time.  The  objective  is  to  avoid  costly 
changes  during  production  or  even  more  costly  field 
retrofits.  The  latter  possibility  might  involve  installa¬ 
tion  teams  waiting  unproductivcly  at  sites  while  design 
problems  are  diagnosed  and  solutions  generated. 

Design  verification  should  start  early  in  the  concep¬ 
tual  phase  of  the  design  and  should  be  continued,  as 
necessary,  until  field  implementation  starts.  During  this 
period,  the  key  or  critical  aspects  of  the  design  should 
be  verified,  checked,  and  tested  to  remove  as  much 
uncertainty  as  possible  in  the  final  design.  In  an  air  de¬ 
fense  system,  for  example,  the  tracking  logic  should  be 
tested  under  a  wide  range  of  realistic  conditions;  solu¬ 
tions  to  interception  equations  should  be  cheeked;  man- 
machine  relationships  and  capabilities  should  be  verified, 
and  so  on. 

Many  different  methods  can  be  employed  for  design 
verification,  ranging  from  complete  simulation  to  live 
tests  with  actual  equipments.  In  the  former,  the  com¬ 
putational  process,  the  operators,  the  environment,  and 
even  the  system  equipments  can  be  simulated;  the 
system  computer  or  some  other  machine  can  be  used 
as  the  vehicle  for  conducting  the  simulation;  and  the 
entire  process  can  be  carried  out  in  a  convenient  time 
scale.  Live  tests,  on  the  other  hand,  can  involve  bread¬ 
board,  prototype,  or  early  production  equipments. 

Simulation  permits  an  investigation  of  performance 
over  a  wide  range  of  conditions,  but  generally  suffers 
from  the  lack  of  realism.  Wc  simulate  what  we  know, 
not  the  unexpected,  and  generally  our  design  has  ac¬ 
counted  for  what  we  know.  Live  tests  are  more  difficult 
and  time  consuming,  and  in  some  instances  are  too 
expensive  or  even  impossible  to  conduct,  as  for  example 
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when  the  test  involves  many  weapons,  ECM,  etc.  In 
many  cases  the  methods  can  be  mixed  or  used  to  sup¬ 
plement  one  another,  as  for  example,  the  use  of  se¬ 
lective  live  tests  to  calibrate  a  set  of  simulations. 

The  methods  employed  will  vary  from  system  to 
system.  The  significant  point  is  that  design  verification 
should  be  considered  as  an  essential  part  of  the  design 
process.  It  should  be  incorporated  into  the  planning 
and  funding  for  the  system.  It  should  be  applied  to  the 
critical  areas  of  the  design  as  early  and  as  realistically 
as  possible.  It  is  one  of  the  few  bits  of  insurance  that 
a  system  engineer  can  buy. 

V.  Hardware 

Computers 

Significant  improvements  have  been  made  in  the 
speed,  capacity,  and  reliability  of  digital  computers 
over  the  last  ten  years.  Today,  many  machines  of  proven 
performance  are  available,  and  they  exhibit  strong 
similarities  in  basic  design  features:  order  codes,  index¬ 
ing  systems,  interrupt  systems,  and  ability  to  handle 
auxiliary  memory  devices  (drums,  tapes,  discs).  The 
chief  differences  are  in  word  lengths,  memory  sizes,  and 
operating  speeds. 

For  the  most  part,  computers  represent  one  of  the 
few  “off-the-shelf”  items  that  the  system  designer  has 
at  his  disposal.  There  are  some  related  hardware  design 
problems,  however,  primarily  in  the  area  of  the  special- 
purpose  in-out  buffers  and  devices  peculiar  to  the  sys¬ 
tem.  As  regards  the  central  computer,  it  is  more  a 
question  of  proper  selection  of  machine  and  configura¬ 
tion  rather  than  design. 

The  critical  consideration  in  the  selection  of  a 
machine  is  that  of  adequate  capability:  storage  capacity, 
in-out  capacity,  and  operating  speed.  It  has  already 
been  mentioned  that  care  must  be  taken  to  allow  excess 
capacity  both  for  contingencies  in  the  original  design  as 
well  as  for  unseen  requirements.  A  machine  that  looks 
just  big  enough  at  the  early  stages  of  design  is  surely 
not  going  to  be  adequate  at  the  end.  A  safety  factor 
of  two  would  not  seem  unreasonable,  particularly  as 
the  cost  of  more  storage  and  higher  speeds  seems  to 
be  decreasing. 

When  comparing  the  required  speed  of  computation 
against  available  machines,  one  should  be  careful  of 
high  computational  speeds  achieved  by  special  machine 
features  requiring  sophisticated  software,  either  in  the 
asscmbly/compilcr  programs  or  in  the  operational  pro¬ 
grams.  Many  machines  can  only  achieve  these  high 
speeds  when  the  program  structure  permits  use  of  the 
special  hardware  features.  As  a  more  general  point,  one 
should  not  attempt  to  economize  on  hardware  by  as¬ 
suming  efficient,  well-written  programs — these  are  both 


difficult  and  costly  to  achieve.  Well-trained,  experienced 
programmers  arc  in  very  short  supply. 

The  second  aspect  of  machine  selection  relates  to 
the  configuration  required  to  achieve  the  desired  re¬ 
liability.  Until  quite  recently,  if  a  premium  was  placed 
on  continuity  of  operation,  a  complete  duplexing  would 
be  required.  This  was  done  with  considerable  success 
in  SAGE.  Two  machines  are  available  at  each  direction 
center,  with  only  one  being  operational  at  a  time.  The 
machines  arc  connected  by  a  drum  through  which  they 
can  communicate,  thereby  permitting  the  second  or 
standby  machine  to  accumulate  the  dynamic  data  base 
required  for  rapid  assumption  of  responsibility.  The  hieh 
reliability  actually  achieved  from  each  machine,  coupled 
with  a  programmed  ability  to  recover  from  most  inter¬ 
mittent  errors,  has  led  to  very  long  mean-times  be¬ 
tween  disabling  system  failures.  Several  years  ago  when 
I  last  checked,  the  mean  free-time  was  about  20  days. 

Today  the  computer  situation  is  changing,  and  the 
selection  of  a  machine  or  configuration  of  machines  has 
taken  on  several  different  aspects.  This  changing  situa¬ 
tion  results  from  the  development  of  modular  machines. 
In  their  first  appearance,  the  modularity  was  restricted 
to  memory  capacity  and  in-out  channel  capacity.  If 
more  of  either  were  needed,  added  modules  could  be 
connected.  Today,  the  modular  concept  has  extended 
to  the  central  processor  itself,  and  the  truly  modern 
machine  design  includes  the  capability  of  employing 
several  processors  operating  in  parallel  and  sharing  the 
available  memory  and  in-out  modules.  This  permits 
what  has  been  termed  “multi-processing,”  with  several 
processors  operating  together  on  a  single  job.  (It  is 
to  be  distinguished  from  “multi-programming,”  in 
which  one  machine  works  on  several  different  tasks.) 

Through  multi-processing,  modular  machines  can 
achieve  very  high  effective  operating  speeds.  However, 
except  for  those  very  real-time  systems  which  might 
require  operating  speeds  beyond  the  capability  of  a 
single  central  processor,  the  advantages  of  modularity 
lie  other  than  in  the  direction  of  high  speeds. 

First,  of  course,  is  the  capability  for  growth  by  ad¬ 
dition  of  modules.  Second,  is  the  high  reliability  which 
can  be  achieved  at  a  relatively  low  price;  a  full  duplex 
is  not  needed,  rather,  only  one  or  two  spare  modules 
are  required  for  each  different  element.  Further,  with 
proper  software  design  it  is  possible  for  individual 
modules  to  fail  without  complete  interference  with  the 
system  operation.  This  has  been  termed  a  “fail-softly” 
or  “fail-gracefully”  characteristic.  Third,  there  is  the 
ability  to  tailor  the  equipment — that  is,  the  number 
or  modules — to  the  situation  or  capacity  required  at 
different  sites.  That  is,  if  a  300-aircraft  control  system 
were  needed  at  one  site,  6  memory  modules  might 
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be  required;  but  at  a  site  requiring  only  50  tracks,  only 
2  modules  would  be  needed. 

Parallel  operation  of  computers  is  employed  in  the 
NTDS,  or  Navy  Tactical  Data  System,  in  which  a  three- 
processor  configuration  is  required  to  do  the  full  fleet 
air  defense  task  on  a  major  ship.  One  machine  does 
tracking,  another  docs  the  intercept  calculations,  and  a 
third  processes  display  information  and  performs  mis¬ 
cellaneous  tasks.  Under  less  than  full-load  conditions, 
two  machines  can  be  used  with  different  programs  to 
perform  the  same  job  at  lower  capacity.  The  third 
machine  is  then  available  for  maintenance  or  other 
support  tasks.  It  is  also  possible  to  operate  at  a  one- 
machine  level,  and  this  is  done  on  smaller  ships  not 
requiring  intercept  control  capabilities. 

The  NTDS  design,  however,  does  not  utilize  a  com¬ 
mon  memory;  rather,  the  individual  computers  exchange 
information  via  in-out  channels.  The  French  STRIDA 
If  air  defense  system  employs  multiple  computers,  each 
permanently  assigned  to  specific  tasks  but  sharing  some 
common  memory. 

The  full,  multi-processing  potential  of  the  recent 
modular  machines  has  not  yet  been  realized  in  actual 
systems.  The  FAA  design  for  an  improved  air  traffic 
eontrol  system — the  National  Airspaee  System — will 
ultimately  incorporate  a  full  modular  multi-processing 
capability,  but  initial  system  implementation  will  be 
along  the  more  conventional  lines. 

The  software  design  problems  associated  with  exploit¬ 
ing  a  full  multi-processing  capability  are  quite  signifi¬ 
cant.  They  include  the  problem  of  breaking  down  the 
program  into  small,  relatively  independent  parts,  and 
the  design  of  adequate  executive  routines  to  handle  the 
traffic,  to  sense  modular  malfunctions,  and  to  manage 
the  assignments  and  switching  of  modules.  There  are 
related  hardware  problems.  It  may  be  some  years  yet 
before  it  is  desirable  (or  necessary)  to  spend  the  money 
and  effort  to  solve  these  problems  as  long  as  the  more 
conventional  high-speed  sequential  machines  in  a  sim¬ 
ple  duplex  configuration  are  adequate  to  the  tasks. 

Display  consoles 

The  situation  with  display  consoles  is  quite  the  op¬ 
posite  from  that  of  computers.  There  are  relatively  few 
‘‘off-the-shelf”  equipments,  there  have  been  only  limited 
advances  in  performance  or  cost  over  the  past  ten  years, 
and  there  arc  few  systems  in  which  the  user  is  satisfied 
with  his  display  consoles. 

The  situation  is  aggravated  by  the  fact  that  it  is  ex¬ 
tremely  difficult  to  reach  an  agreement  on  requirements. 
The  display  eonsole  is  one  part  of  the  system  in  which 
the  operator  normally  takes  a  strong  interest,  and  he 
naturally  desires  the  utmost  in  performance.  Unfortu¬ 
nately,  however,  he  is  an  easy  prey  of  the  “broehu reman- 


ship”  technique  since  he  does  not  always  appreciate  the 
difference  between  a  paper  design  and  a  working  model, 
or  between  a  laboratory  model  and  a  field-maintained 
production  unit.  The  net  result,  then,  is  that  his  re¬ 
quirements  are  very  high:  much  information;  rapidly 
changing  alphanumeric  characters;  flicker-free  presen¬ 
tation;  a  bright  display  under  high  illumination  levels; 
and  possibly  even  color. 

The  designer  finds  it  difficult  to  challenge  the  need, 
and  the  problems  of  complexity,  cost,  and  maintain¬ 
ability  are  not  received  sympathetically  by  the  operator. 
It  is  unfortunate  that  it  is  not  generally  possible  to 
quickly  put  together  and  demonstrate  various  display 
capabilities  so  that  the  advantage  or  need  of  various 
features  could  be  objectively  determined  before  pro¬ 
ceeding  with  production  equipments. 

Display  consoles,  then,  represent  a  most  difficult 
problem  for  the  designer.  While  it  may  be  impossible  to 
completely  satisfy  the  operator,  he  must  be  given  a 
usable  and  reliable  display.  Economic  factors  cannot 
be  ignored,  particularly  at  the  present  production  costs 
of  $30,000  to  over  $100,000  per  console.  Added  points 
of  caution  or  consideration  include: 

•  Whenever  possible,  a  standard  console  design 
should  be  adopted,  with  very  minor  modifications 
for  specific  operating  positions. 

•  Character  or  symbol  sizes  should  be  kept  as 
small  as  possible  within  the  limits  of  legibility. 
Special  provisions  (perhaps  in  the  software)  may 
be  required  to  prevent  overlap  of  symbology,  as 
might  result  from  adjacent  aircraft  tracks. 

•  The  ambient  lighting  environment  should  be 
carefully  understood  or  designed,  particularly  as 
this  may  cause  reflections  on  the  scope  face. 

•  The  general  requirement  for  large  viewing  surfaces 
should  be  balanced  against  smaller  viewing  areas 
coupled  with  a  capability  for  off-centering  and 
expansion. 

•  The  merits  of  alphanumeric  characters  as  op¬ 
posed  to  a  limited  symbolic  capability  should  be 
matched  against  the  differences  in  equipment 
complexity  and  cost. 

•  In  addition  to  the  console-operator  interface, 
attention  should  be  given  to  the  computer-con¬ 
sole  interface.  The  display  console  is  usually  one 
of  the  earliest  identified  subsystems,  but  its  design 
must  carefully  consider  the  interface  with  the 
computer,  and  particularly  with  the  computer 
program.  The  tradeoff  between  the  amount  of 
computer  programming  involved  in  display 
formatting  and  the  complexity  of  the  eonsole  itself 
is  not  an  easy  one  to  make. 

•  Requirements  for  background  or  geography  dis¬ 
plays  must  be  considered. 
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Large  screen  displays 

The  situation  with  large  screen  displays  is  not  very 
different  from  that  of  consoles  in  that  requirements  arc 
difficult  to  resolve,  the  user  generally  adds  a  multiple- 
color  requirement,  and  the  available  systems  arc  limited. 
A  large  number  of  techniques  ranging  from  dry  process¬ 
es  to  wet  processes,  and  from  xerographic  techniques  to 
theater  TV  techniques,  have  been  proposed,  but  only  a 
few  have  yet  reached  the  stage  of  working  systems. 

At  the  present  time,  silver  halide  systems  involving 
projection  of  images  photographed  from  the  face  of  a 
CRT  are  available  with  processing  times  of  about  10 
seconds.  The  maintenance  problems  of  such  systems 
seem  under  control.  Four-color  systems  using  separate 
projectors  or  filters  are  used  in  a  number  of  systems; 
color  mixing  has  not  generally  been  reduced  to  field 
practice. 

Beyond  reliability,  a  major  problem  is  that  of  bright¬ 
ness,  since  the  command  posts  in  which  these  projected 
displays  arc  used  are  normally  kept  at  high  illumination 
levels. 

Input  devices 

Special-purpose  action  switches  and  general-purpose 
keyboards  arc  the  principal  devices  by  which  operators 
insert  data  into  the  system  or  influence  the  processing. 
Switches  are  quick  and  easy  to  use,  but  pose  a  problem 
when  many  different  possible  actions  arc  required.  In 
order  to  retain  the  simplicity  of  switch  inputs  while 
keeping  the  number  of  switches  within  reasonable 
bounds,  some  recent  designs  have  utilized  switches  cap¬ 
able  of  performing  several  different  functions  by  cither 
manual  or  automatic  label  changes. 

For  inputs  requiring  more  flexibility,  particularly 
those  involving  variable-length  items  or  alphanumeric 
messages,  a  keyboard  is  required.  The  problems  of 
format  and  content  errors  have  resulted  in  sophisticated 
equipments,  generally  termed  “message  composers,’'  to 
assist  the  operator.  Other  designs  have  relied  upon  the 
computer  to  help  the  operator  in  the  composition- 
formatting-error  detection-error  correction  process  by  a 
feedback  of  computer  confirmation  or  error  information 
on  printers.  Standard  teletype  machines  can  then  be 
used  as  the  input  device. 

For  operators  working  on  consoles  with  plan-position 
displays,  an  ability  to  designate  or  point  out  selected 
positions  can  be  achieved  by  a  photoelectric  cell  “light 
gun"  or  “light  pencil"  or  by  movable  display  circle 
or  “hook"  controlled  by  a  “tracking  ball"  (or  “joy¬ 


stick").  Both  of  these  types  of  devices  can  also  be  used 
as  a  more  general  input  technique  whereby  the  op¬ 
erator  may  select  and  designate  quickly  with  his 
“pointer"  among  a  number  of  alternatives  presented  in 
an  alphanumeric  message  form  on  the  display  console, 
this  technique,  an  extension  of  some  earlier  work  on 
“electronic  typewriters,”  appears  to  have  considerable 
promise  in  command  systems  requiring  rapid,  lengthy, 
and  flexible  man-machine  exchanges. 

VI.  Software 

The  computer  programs,  or  software,  arc  of  central 
importance,  since  they  direct  the  data  processing 
equipment,  and  hence  the  operation  of  the  system.  They 
usually  represent  a  sizable  fraction  of  the  total  system 
design  and  development  effort,  and  their  cost  in  most 
systems  is  comparable  to  that  of  the  computers  them¬ 
selves. 

Nevertheless,  the  results  achieved  in  this  area  leave 
much  to  be  desired.  The  universal  experience  is  that 
the  magnitude  of  the  computer  programming  activity, 
both  in  time  and  effort,  is  grossly  underestimated. 
Further,  the  inherent  potential  of  these  programs  for 
ease  of  modification  has  not  been  realized;  in  practice, 
and  for  a  variety  of  reasons,  the  operational  pro¬ 
grams  have  not  been  flexible,  and  to  change  them  has 
been  costly  and  time  consuming. 

A  first  design  problem,  then,  is  to  recognize  the  total 
size  and  scope  of  the  programming  task.  In  addition  to 
the  operational  program  itself — which,  as  noted  earlier, 
should  contain  performance  monitoring,  data  recording, 
checkout  features,  etc. — it  is  necessary  to  plan  for  the 
other  programs  required  in  the  software  production, 
checkout,  test,  and  installation.  These  additional  pro¬ 
grams  can  be  categorized  as  follows: 

•  Utility  programs  necessary  for  fabricating  the 
operational  program.  These  cover  such  areas  as 
assembly,  diagnostics,  tape  handling  and  process¬ 
ing,  analysis,  documentation,  and  control.  As 
noted  earlier,  hopefully  many  of  these  programs 
can  be  procured  with  the  computer  itself. 

•  Support  programs  used  to  expedite  the  testing  of 
the  operational  program  during  fabrication.  These 
would  provide  the  parameter  and  other  data  in¬ 
puts  required  for  such  program  testing  phases 
as  parameter  testing  of  sub-programs,  assembly 
testing  of  groups  of  subprograms,  program  shake- 
down,  and  system  shakedown. 

•  The  test  data  reduction  programs  required  to 
evaluate  the  performance  of  the  operational 
program  during  test  phases. 
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•  The  operational  data  analysis  programs  required 
to  support  evaluation  of  system  operation  on  a 
longer  term  basis. 

The  number  and  size  of  the  programs  in  these 
categories  vary  with  each  different  system.  In  SAGE, 
which  pioneered  much  of  the  work  on  the  types  of 
utility,  support,  and  data  reduction  programs  needed 
for  real-time  systems,  a  very  large  effort  was  expended 
on  the  nonoperational  programs.  Further,  the  rates 
at  which  programs  in  the  different  categories  can  be 
designed  and  produced  vary  significantly. 

Production  rates  on  the  operational  program,  in 
particular,  may  be  an  order  of  magnitude  less  than  on 
conventional  scientific  and  engineering  programs.  Be¬ 
cause  of  its  size,  the  operational  program  must  usually 
be  broken  down  into  smaller  pieces  for  individuals  to 
work  on.  This  complicates  both  the  design  problem  and 
the  subsequent  assembly  and  checkout  of  the  various 
subprograms.  Because  the  subprograms  may  refer  to  or 
change  common  items  of  stored  information,  and  be¬ 
cause  it  is  not  always  easy  or  desirable  to  freeze  the 
design  of  the  data  tables  at  an  early  stage,  special  tech¬ 
niques — notably  the  use  of  common  symbolic  tags  and 
compools — have  been  established.  This,  then,  requires 
that  sophisticated  special-purpose  compilers  must  be 
used.  Further,  since  real-time  data  processing  generally 
requires  exploiting  the  speed  of  the  machine,  a  good 
deal  of  machine-language  programming  may  be  re¬ 
quired. 

The  production  rates  of  operational  programs  arc 
further  lowered  by  the  extensive  checkout  required. 
Each  subprogram  must  be  tested  under  a  variety  of 
input  conditions — this  is  often  called  parameter  testing 
— and  then  the  programs  must  be  assembled  together 
and  checks  made  on  the  continuity  of  operation.  Finally, 
the  entire  program  should  be  tested  under  a  wide 
variety  of  operating  conditions. 

To  illustrate  these  points,  Table  1  shows  some  esti¬ 
mates  on  program  sizes  and  efforts  for  a  proposed  real¬ 
time  data  processing  system  for  which  considerable  re¬ 
lated  experience  is  available.  All  programs  were  assumed 
to  be  produced  with  a  modified  assembly  program,  with 
the  exception  of  the  bulk  of  the  data  reduction  and 
analysis  programs  which  would  be  written  for  a  separate 
commercial  machine  using  FORTRAN.  The  “produc¬ 
tion  rate'7  includes  all  activities  beginning  with  the  pro¬ 
gram  design  activities  (given  program  performance 
specifications)  and  terminating  with  the  handover  of  a 
tested  program,  including  card  decks,  listings,  design 
and  coding  specifications,  and  manuals. 


Table  I 

Program  Production  Estimates 

Production  Production 


Category 

Size* 

Ratet 

Effort  t 

Operational 

50,000 

70 

700 

Utility 

40,000 

250 

160 

Support 

Test  Data 

40,000 

250 

160 

Reduction 

120,000 

1200 

Operational  Data 

20,000 

250 

180 

Analysis 

30,000 

250 

120 

TOTALS 

300,000 

1320 

"Single  Address  Instruclions 
■^Instructions  Per  Man-Month 
t  Man-Months 

In  this  table,  it  was  assumed  that  the  computer  came 
with  the  normal  repertoire  of  assembly,  loader,  trap, 
and  trace  programs,  and  that  the  special-purpose  equip¬ 
ment  test  programs  required  to  check  out  and  test 
various  equipment  subsystems — display  consoles,  input/ 
output  equipments,  data  links,  etc. — would  be  provided 
by  equipment  suppliers. 

Tabic  I  demonstrates  two  points  of  common  ex¬ 
perience: 

•  The  size  of  the  supporting  programs  is  generally 
greater  than  the  operational  program  by  a  factor 
of  2  to  5. 

•  The  effort  in  producing  the  supporting  programs 
generally  equals  and  may  exceed  the  effort  on 
the  operational  program. 

As  an  added  word  of  caution,  it  should  be  noted  that 
estimators  of  program  sizes  arc  traditionally  optimistic 
(they  base  their  estimates  on  what  they  think  they 
themselves  could  do),  yet  the  program  is  inevitably 
written  largely  by  relatively  new  and  unproductive 
programmers  (experienced  programmers  generally 
graduate  to  writing  sophisticated  compilers  or  they  be¬ 
come  managers). 

The  key  lessons  in  this  size  and  effort  area  are: 

•  Identify  and  plan  for  all  necessary  computer  pro¬ 
grams  at  the  earliest  date. 

•  Do  not  underestimate  the  checkout  and  documen¬ 
tation  activities. 

•  Do  not  assume  program  production  rates  nor¬ 
mally  achieved  in  scientific  or  business  data 
processing  programs. 

•  Expect  reduced  production  rates  as  the  magnitude 
of  the  operational  program  and  number  of  sub¬ 
programs  grow. 

A  second  major  software  problem  is  the  organization 
of  the  operational  program  into  relatively  independent 
modules  or  subprograms.  For  example,  should  display 
generation  functions  be  distributed  among  the  various 
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subprograms  which  may  affect  displays — tracking,  iden¬ 
tification,  weapons  assignment,  etc. — or  should  they 
be  grouped  into  one  display  generation  subprogram? 
Economy  of  storage  and  operating  time  points  to  con¬ 
solidation;  flexibility  for  change  points  to  distribution. 

Once  the  functions  of  each  subprogram  module  are 
determined,  it  becomes  necessary  to  define  the  precise 
inputs  and  outputs,  that  is,  the  transfer  function  of 
the  module. 

The  fixed  and  dynamic  data  storage  tables  offer 
many  design  choices.  Here,  too,  efficiency  of  storage 
utilization  usually  runs  counter  to  the  design  which  of¬ 
fers  the  greatest  flexibility. 

A  master  or  executive  program  is  normally  required 
to  direct  the  sequential  execution  of  subprograms,  to 
handle  transfers  of  tables  and  other  data  from  the  high 
speed  memory  to  and  from  other  storage  media,  and  to 
handle  the  in-out  and  interrupt  processing.  The  execu¬ 
tive  program  must  be  capable  of  handling  subprograms 
operating  at  different  rates;  some  are  required  periodi¬ 
cally,  some  only  on  demand.  The  executive  program 
must  also  possess  high  flexibility  for  addition  of  new 
subprograms  and  for  modifications  of  program  sequence 
or  periodicity.  In  summary,  it  merits  the  most  careful 
design,  including  consideration  of  the  instrumentation 
and  testing  requirements  imposed  during  program  as¬ 
sembly  and  checkout. 

Throughout  the  software  design,  attention  must  be 
given  to  the  need  for  and  techniques  of  adapting  a 
master  operational  program  for  use  at  different  sites 
with  individual  characteristics  or  parameters. 

During  the  design  process,  attention  should  also  be 
given  to  the  matter  of  response  time,  and,  in  particular, 
to  the  elapsed  time  from  an  operator  input  or  request 
to  the  related  computer  output  or  response.  Three  times 
arc  involved:  recognition  of  the  input  and  routing  to 
the  proper  subprogram,  subprogram  processing,  and  the 
final  output  processing.  Proper  design  of  the  terminal 
equipment  can  minimize  the  initial  and  final  steps.  Some 
difficulty  has  been  experienced  in  several  systems  with 
the  subprogram  processing.  Several  subprograms  may 
be  involved,  and  if  searching  of  lengthy  files  is  required, 
surprisingly  long  response  time — on  the  order  of  min¬ 
utes — can  result.  Priority  processing  of  inputs  and  care¬ 
ful,  and  possbily  redundant,  table  design  may  be  re¬ 
quired. 

As  noted,  case  and  rapidity  of  modification  represent 
outstanding  software  design  problems.  The  current  lead 
times  of  months,  and  possibly  up  to  a  year,  for  modest 
field  changes  arc  clearly  undesirable  and  tend  to  belie 
the  name  of  software.  With  the  availability  of  larger 
and  faster  machines,  it  is  becoming  possible  to  design 
the  program  into  relatively  independent,  although  less 
efficient,  modular  packages.  If  this  is  done,  new'  pro¬ 


gram  modules  can  be  added  and  old  program  modules 
changed  without  requiring  a  major  modification  of  the 
entire  program.  However,  one  should  be  wary  of  going 
too  far  to  this  extreme  where  case  and  rapidity  of 
modification  may  provide  a  temptation  to  unneces¬ 
sarily  modify  the  system. 

A  very  interesting  possibility,  adapted  in  part  from 
business  data  processing,  is  the  development  of  gen¬ 
eral-purpose  programs  or  software.  This  concept  would 
exploit  the  similarity  of  many  functional  processes  in 
the  operational  programs,  particularly  in  command  sys¬ 
tems  involving  data  manipulation.  For  example,  the 
functions  of  file  updating,  file  retrieval,  message  proc¬ 
essing,  display  make-up,  or  report  generation  could  be 
programmed  in  a  very  general  form,  and  then  adapted 
to  specific  applications  by  supplying  the  detailed  de¬ 
scriptors  for  the  files  and  the  data.  An  extension  of  this 
technique  would  then  permit  on-line  compilation  of 
programs  by  operational  personnel  to  achieve  specific 
needs.  Initial  research  in  these  areas  appears  to  offer 
promising  results,  although  not  without  large  require¬ 
ments  for  computer  storage  and  operating  time. 

Finally,  the  initial  program  production  effort  must 
be  carried  out  with  due  regard  to  the  subsequent  pro¬ 
duction  effort.  Current  practice  and  direction  is  for 
assumption  of  these  on-going  efforts  by  military  per¬ 
sonnel.  This  places  stringent  demands  on  the  documen¬ 
tation  of  the  initial  program,  may  influence  the  choice 
between  problem-oriented  and  machine-oriented  com¬ 
pilers,  and  requires  integration  of  key  military  personnel 
in  the  initial  design  and  production  activity. 

VII.  Testware  and  testing 

Throughout  the  system  design  and  engineering  ef¬ 
fort,  adequate  attention  must  be  given  to  the  testware — - 
the  plans,  equipments,  computer  programs,  and  pro¬ 
cedures  and  techniques  associated  with  the  system  test 
activities. 

Early  definition  of  the  phases  of  the  system  test 
activity  is  essential.  These  phases  might  include  design 
verification,  the  so-called  Category  1  and  II  tests  relating 
to  acceptance  and  test  at  the  manufacturer’s  plan  and 
an  initial  field  site,  implementation  tests  at  successive 
field  site,  a  full-scale  operational  evaluation,  and  follow- 
experimental  tests  leading  to  improvements. 

Again  the  names  are  not  significant,  and  the  nature 
and  type  of  testing  will  vary  from  system  to  system. 
The  significant  point  is  that  a  decision  as  to  the  nature 
and  types  of  test  activity  be  made  at  a  sufficiently  early 
date  to  permit  a  determination  of  the  requirements  for 
special  equipments  and  Facilities,  special  computer  pro¬ 
grams,  test  teams,  operational  personnel,  special  flight 
tests,  computer  time,  etc.  This  should  then  be  followed 
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up  with  the  necessary  plans,  schedules,  contracts,  and 
other  arrangements. 

The  manpower  requirements  for  planning,  designing, 
documenting,  conducting,  and  analyzing  a  test  program 
in  a  large  system  can  be  sizable,  and  suitable  provisions 
should  be  made.  The  use  of  a  separate  contractor  not 
associated  with  the  design  or  production  of  hardware 
or  software  has  considerable  merit  from  the  point  of 
view  of  objectivity.  It  will  also  go  a  long  way  towards 
assuring  that  sufficient  attention  is  given  to  the  entire 
tcstwarc  problem.  Such  a  separate  and  independent  test 
contractor  was  used  in  SAGE  with  considerable  success. 

Of  particular  importance  is  an  early  recognition  that 
the  test  plans  and  schedules  must  take  into  account 
certain  facts  of  system  life.  Not  all  tests  will  be  eon- 
ducted  as  originally  planned;  tests  will  be  delayed  and 
disrupted  for  a  variety  of  reasons;  and  tests  will  un¬ 
cover  errors  and  deficiencies  that  will  require  sub¬ 
stantial  amounts  of  time  for  redesign  and  correction. 
Optimistic  test  schedules  are  not  realistic. 

In  a  multi-site  system  or  one  involving  many  re¬ 
mote  input-output  locations  or  connections,  the  se¬ 
quential  phasing  of  the  test  activity  requires  special 
attention.  In  all  systems,  a  carefully  generated,  methodi¬ 
cal  plan  for  the  availability  and  test  of  the  subsystems 
and  various  sites  is  critical.  To  the  greatest  degree  pos¬ 
sible,  there  should  be  a  capability  to  test  and  verify 
subsystem  performance  independently  of  the  rest  of  the 
system.  For  this  purpose,  instrumentation — in  the  form 
of  equipments  and  programs — must  be  provided. 

The  test  activity  is  only  meaningful  if  there  arc  cri¬ 
teria  against  which  measured  performance  can  be 
checked,  and  if  the  system  inputs  and  environment  can 
be  controlled  when  the  performance  is  measured.  It  is 
necessary  to  determine  what  is  to  be  measured,  under 
what  conditions  this  is  to  be  done,  and  how  much  data 
is  to  be  collected  (the  size  of  the  sample).  None  of  this 
is  easy  to  do.  Finally,  the  level  of  acceptable  perform¬ 
ance  must  be  decided.  This  is  a  particularly  difficult 
but  necessary  task.  It  generally  requires  much  com¬ 
promise  and  agreement  among  the  designer,  tester,  and 
ultimate  user. 

Verification  of  test  procedures  and  determination  of 
the  test  measures  should  be  conducted  as  early  as  pos¬ 
sible,  and  can  be  one  of  the  objectives  of  a  design 
verification  activity.  In  a  multi-site  system,  Category  11 
tests  can  provide  the  performance  measures  against 
which  successive  sites  will  be  checked. 

In  designing  the  test  activity,  consideration  must  be 
given  to  the  availability  of  trained  operational  person¬ 
nel.  Without  adequately  trained  operators,  it  may  be 
impossible  to  conduct  useful  tests.  A  corollary  problem, 
of  course,  is  the  early  availability  of  trained  maintenance 
personnel. 


Finally,  conducting  tests  at  a  site  while  maintaining 
manual  operations  may  pose  severe  problems.  In  an  air 
defense  system,  for  example,  it  is  extremely  difficult  to 
share  the  use  of  seareh,  beacon,  and  height-finding 
radars.  These  potential  problems — and  suitable  opera¬ 
tional  procedures  or  equipment  modifications — also  re¬ 
quire  early  consideration. 

All  in  all,  the  experience  to  date  has  been  that  ade¬ 
quate  provisions  for  the  testware  are  not  usually  made, 
resulting  in  grossly  extended  and  costly  test  periods. 
The  technical  problems  of  defining  adequate  test  cri¬ 
teria  and  obtaining  system  inputs  that  are  sufficiently 
controlled  (or  at  least  known)  to  allow  meaningful 
measurements  are  not  yet  solved.  We  still  do  not  ex¬ 
pend  sufficient  time  and  effort  in  planning  and  prepar¬ 
ing  for  the  test  activity  well  before  it  starts. 

CONCLUSION 

As  noted,  it  is  perhaps  too  early  for  an  objective  assess¬ 
ment  of  the  development  of  the  first  generation  of 
military  real-time  data  processing  systems.  From  the 
limited  perspective  of  participating  engineers  and  de¬ 
signers,  this  paper  has  attempted  to  summarize  the  key 
system  engineering  lessons  of  this  experience. 

It  must  be  recognized  that  these  lessons  are  not 
necessarily  unique  to  military  real-time  data  processing 
systems,  but  may  generally  be  true  of  all  large  system 
endeavors.  This  realization,  in  fact,  may  be  the  most 
signifieant  conclusion  one  could  reach.  Beyond  that 
point,  however,  I  should  like  to  add  further  emphasis 
to  several  items: 

•  It  is  important  to  develop  a  proper  appreciation 
of  the  full  magnitude  and  scope  of  the  effort  at 
the  start.  The  system  engineer  must  take  a  very 
broad  point  of  view,  far  beyond  the  confines  of 
operational  equipments  and  programs.  In  par¬ 
ticular,  the  design  must  make  provisions  for  a 
wide  range  of  essential  support  functions  and 
activities  at  the  site  and  system  level. 

•  The  management  structure  and  assignment  of 
responsiuilities  for  system  acquisition  can  have 
a  profound  effect  on  the  final  product.  Opera¬ 
tional  representation  and  inputs  are  essential; 
strong  central  design  control  is  required  regard¬ 
less  of  contractual  mechanisms  and  responsioil- 
itics. 

•  The  proper  matching  of  man-machine  capabilities 
and  the  provision  of  adequate  capacity  and  flexi¬ 
bility  for  change  and  growth  are  outstanding  sys¬ 
tem  design  problems. 

•  Documentation  and  design  verification  are  vital 
elements  of  the  design  process. 

•  Software  problems  are  invariably  underestimated, 
and  much  remains  to  be  done  to  realize  the  in- 


System  Engineering  Experience  213 


herent  potential  of  these  programs  for  ease  and 
rapidity  of  modification. 

Testware  design  merits  comparable  attention  with 
hardware  and  software. 


Finally,  and  perhaps  needless  to  say,  system  engineer¬ 
ing  is  a  necessary  function  in  the  system  development 
and  acquisition  process.  It  does  not  “just  happen”;  it 
must  be  carefully  planned  and  deliberately  applied. 


Software  lessons  learned  in  military  command  systems 


by  John  R.  Ottina 

System  Development  Corporation 
Santa  Monica,  California 


INTRODUCTION 

The  first  decade  of  implementing  computer  programs 
for  military  command  systems  has  concluded. 
Looking  back  over  this  period,  we  can  identify  many 
major  accomplishments  and  can  enumerate  many 
technical  and  management  problems  which  have 
been  solved.  The  first  section  of  this  paper  presents 
a  review  of  selected  management/development  prob¬ 
lems  t hiit  are  successfully  being  dealt  with.  The 
second  section  discusses  three  other  problems  which 
have  not  yet  been  fully  resolved  and  proposes  tech¬ 
niques  for  solving  them. 

Selected  review'  of  lessons  learned 

Early  efforts  in  implementing  military  command 
systems  taught  the  developers  several  lessons  which 
are  now  well  recognized.  Further  activities  are  in 
progress  in  the  military  command  system  community 
to  gather  data,  define  procedures,  set  standards, 
etc.,  in  eaeh  of  these  known  problem  areas  which 
will  further  refine  the  solution.  A  summary  of  some 
of  these  lessons  follows.  The  problem  areas  re¬ 
viewed  are  not  all  inclusive  of  those  faced  or  for 
which  a  solution  was  constructed,  but  are  only  a 
subset  which  to  the  author  seem  to  be  of  particular 
importance. 

Cost  estimates 

Perhaps,  one  of  the  earliest  lessons  learned  was 
that  the  initial  estimates  of  the  software  design 
and  development  were  typically  over-optimistic. 
The  complexity  of  the  task  and  the  number  of  people 
required  to  accomplish  it  were  much  greater  than  had 
been  at  first  perceived.  The  value  of  thorough  analy¬ 
sis  and  planning  prior  to  the  development  of  the 
command  system  in  order  to  arrive  at  realistic 
estimates  of  cost,  manpower,  and  schedules  was 
quickly  recognized.  Today  many  attempts  are  being 
made  to  further  validate  cost  estimation  techniques, 
and  as  a  whole  the  reliability  and  validity  of  schedules 
and  gross  manpower  estimates  are  steadily  improving. 
The  need  for  full  understanding  of  the  task  to  be 
accomplished  and  for  the  publication  of  work  plans 


before  making  commitments  as  to  delivery  schedules, 
costs  and  manpower  is  well  recognized. 

Timely  initiation  of  software  effort 

The  developers  of  a  command  system  in  many 
instances  realized  too  late  that  they  had  tailed  to 
initiate  software  design  early  enough  in  the  develop¬ 
ment  of  the  system.  Many  of  the  early  systems  found 
that  failure  to  initiate  software  design  early  enough 
caused  the  total  system  to  be  delayed  and  that  the 
critical  path,  in  many  cases,  was  not  hardware  but 
software.  The  realization  of  the  important  part  that 
software  plays  in  the  command  system  is  now  wide¬ 
spread  and  not  only  are  schedule  delays  from  this 
cause  being  minimized,  but  in  addition  many  trade¬ 
offs  are  made  possible  between  software  and  hard¬ 
ware  early  in  the  design  and  development  phases. 

Establishment  of  requirements  and  control 

Early  efforts  in  the  development  of  command  sys¬ 
tems  frequently  ran  into  difficulties  because  both 
the  software  developer  and  the  procuring  agency 
failed  to  gain  a  full  understanding  of  the  system 
to  be  implemented.  Furthermore  it  was  not  recog¬ 
nized  that  the  environment  of  a  command  system 
changes  throughout  the  development  cycle.  The 
developer  and  the  procuring  agency  failed  to  es¬ 
tablish  techniques  which  would  allow  concurrence 
on  the  system  requirements  and  the  initiation  of 
procedures  for  processing  future  changes. 

Today  the  picture  is  much  improved.  The  last 
few  years  have  seen  major  accomplishments  in  these 
areas.  Specifications  for  the  design  requirements 
for  the  command  control  system  are  now  produced 
by  the  procuring  agency.  Further  detailed  spec¬ 
ifications  are  prepared  by  the  developer  and  are 
reviewed  and  concurred  upon  before  the  actual 
production  of  the  program  begins.  I  hese  specifica¬ 
tions  form  the  basis  for  the  control  of  changes. 
Furthermore,  the  development  of  a  series  of  reviews 
and  boards  or  committees  who  constantly  monitor 
and  approve  changes  to  the  system  has  provided 
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sound  procedures  for  controlling  the  configuration 
of  the  system. 

Software  design  guidance 

Some  of  the  initial  efforts  to  develop  command  sys¬ 
tems  ran  into  difficulties  because  of  failure  to  in¬ 
clude  sufficient  guidance  at  the  outset  for  the  design 
of  the  software  system  itself.  Far  too  much  was  left 
to  the  discretion  of  the  individual  constructing  each 
of  the  separate  computer  programs.  This  error  be¬ 
came  apparent  when  the  pieces  of  the  system  were  put 
together.  Not  only  were  there  inconsistencies  and 
incompatibilities  between  portions  of  the  system 
that  were  developed  individually,  but  the  system, 
when  its  elements  were  integrated,  had  large  holes 
in  it  and  would  not  function  properly.  The  crys¬ 
tallization  of  design  concepts  and  principles  in 
the  form  of  specifications  has  helped  to  eliminate 
many  of  these  problems.  These  specifications  have 
also  more  readily  permitted  the  software  developer 
to  establish  for  his  work,  detailed  and  coordinated 
programming  standards,  common  terminology  and 
program  conventions. 

Documentation 

Software  development  does  not  only  mean  writing 
computer  programs,  but  includes  a  requirement  for 
complete  and  accurate  documentation.  In  the  early 
days  of  system  development  this  need  was  over¬ 
looked  and  no  requirements  were  levied  for  doc¬ 
umentation  of  the  program  system,  its  operation, 
its  modification,  etc.  Today  detailed  program  spec¬ 
ifications,  users  manuals,  flow  charts  and  a  wide 
range  of  related  documents  are  recognized  as  being 
required  for  system  implementation,  operation  and 
maintenance.  In  addition,  standards  are  now  being 
produced  by  the  military  which  establish  the  require¬ 
ments  for  uniform  documentation  in  all  command 
systems. 

Quality  control 

The  failure  to  include  measures  for  controlling  and 
verifying  the  quality  of  the  command  system  soft¬ 
ware  during  its  development  led  to  a  great  deal  of 
trouble.  Developers  and  procuring  agencies  even¬ 
tually  came  to  recognize  this  failure  and  have  taken 
steps  to  remedy  it.  A  range  of  in-development  tests, 
test  standards,  criteria  for  acceptance,  demonstra¬ 
tions,  etc.,  has  been  implemented  as  an  integral  part 
of  command  system  development.  These  measures 
have  provided  increased  quality  of  the  software  prod¬ 
ucts,  programs,  documentation,  etc.  Some  form  of 
quality  control  has  become  a  recognized  requirement 
by  both  the  developer  and  procuring  agency.  Several 
programs  of  test  criteria,  test  standards,  etc.,  are 
being  investigated  by  the  community  at  large. 


Experienced  personnel 

A  decade  ago  one  of  the  most  critical  problems  in 
system  development  was  shortage  of  trained,  ex¬ 
perienced  programmers,  designers,  implementation 
specialists,  etc.,  to  do  the  software  task.  Not  only 
were  there  insufficient  numbers  of  these  people,  but 
there  was  little  recognition  of  the  need  to  involve 
them  throughout  all  phases  of  the  development  of 
the  system.  Ten  years  of  working  with  computers 
and  command  systems  have  changed  all  this  and 
there  now  exists  a  rather  large  number  of  experienced 
professions  who  are  drawn  upon  to  face  the  prob¬ 
lems  before  us.  This  is  not  to  say  that  the  supply 
of  these  people  is  adequate  or  that  there  won't  be 
serious  shortages  of  trained  people  in  the  future. 
In  addition,  the  need  for  involvement  of  software 
oriented  people  through  all  phases  of  the  develop¬ 
ment  of  the  command  system  is  well  recognized. 
The  software  developer  knows  as  a  matter  of  routine 
that  he  must  continue  to  train  and  add  to  this  limited 
people  resource. 

User’s  role 

For  all  the  advancement  in  the  technology  of  auto¬ 
mation,  the  developers  of  early  command  systems 
failed  to  pay  sufficient  attention  to  the  human  ele¬ 
ment  in  the  system.  Complex  language,  techniques, 
and  methods  were  used  which  were  strange  and  in¬ 
comprehensible  to  the  user.  Recognition  of  this 
failure  has  seen  the  user  become  more  involved  in 
the  development  phase  and  the  developer  more 
cognizant  that  his  product  was  meant  to  be  used. 
In  particular  the  developer  has  implemented  human 
engineering  techniques,  user  documentation  and  in¬ 
struction  or  training  in  the  use  of  the  system.  This 
application  of  the  systems  approach  from  both  ends 
of  the  development  process  has  resulted  in  a  nar¬ 
rowing  of  the  gap  which  would  otherwise  exist  be¬ 
tween  the  designer  and  user  of  the  system. 

Suggested  solutions  to  other  problems 

The  techniques  employed  to  solve  the  problems 
described  in  the  previous  section  have  been  dem¬ 
onstrated,  and  are  now  well  known  and  routinely 
employed.  Continued  research  and  refinement  in 
most  areas  is  in  progress.  There  exists,  however, 
a  second  class  of  problems  for  which  the  proposed 
solutions  are  more  obscure,  unproved  or  controver¬ 
sial.  Three  examples  of  this  class  of  problems  are 
discussed  in  the  succeeding  sections.  For  each  prob¬ 
lem  a  technique  which  has  been  successfully  em¬ 
ployed  is  proposed  as  a  solution. 

Circumventing  the  unavailability 
of  non-functional  software 

Typically  the  computer  is  available  several  months 
before  the  supporting  software.  One  of  the  critical 
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problems  in  the  development  of  the  operational  com¬ 
puter  program  is  unavailability  of  this  non-functional 
software,  Non-funetional  software,  that  is,  utility 
and  support  computer  programs  that  are  required 
by  the  developer  to  construct  the  operational  com¬ 
puter  program,  must  be  written  in  advance  of  the 
operational  program. 

The  problem  facing  the  developer  of  command  sys¬ 
tems  who  encounters  this  situation  is  how  to  proceed 
without  the  non-functional  software.  The  popular 
solution  attempts  to  identify  those  portions  of  the 
non-functional  software  that  are  required  earliest 
in  the  development  of  the  system  and  to  commit 
resources  to  accomplish  those  tasks.  As  a  result, 
the  nonfunctional  software  is  implemented  on  a  piece¬ 
meal,  incremental  basis.  In  most  cases,  this  approach 
is  adequate  but  it  does  not  eliminate  delay,  and  man¬ 
agers  have  been  led  to  look  for  more  satisfactory 
solutions.  An  alternative  solution  to  the  problem  is 
the  use  of  nonfunctional  software  which  has  been 
constructed  in  higher  order  language  for  a  prior  ap¬ 
plication  on  a  different  computer. 

To  make  this  nonfunctional  software  usable  for  the 
new  computer,  theoretically  all  that  needs  to  be  done 
is  to  modify  the  portion  of  the  compiler  that  converts 
the  higher  order  language  or  the  intermediate  language 
to  the  binary  representation  required  by  the  new 
computer.  This  can  be  accomplished  in  two  stages. 
Stage  one  is  to  modify  the  compiler  to  operate  on  the 
original  computer  as  before,  but  to  produce  the  binary 
code  acceptable  to  the  new  computer.  Stage  two  is  to 
use  the  modified  compiler  developed  in  stage  one  to 
compile  itself,  thereby  obtaining  a  “new”  compiler 
which  now  operates  on  the  new  computer.  We  now  go 
on  to  recompile  the  remaining  portions  of  the  non¬ 
functional  software  with  this  “newly”  constructed 
compiler.  In  actual  practice  the  compiler  will  require 
modification  as  there  will  undoubtedly  be  some  unique 
aspects  of  the  input/output  function  of  the  computer 
that  must  be  dealt  with.  In  addition  the  code  generated 
will  not  be  optimum.  This  latter  factor  can  be  dealt 
with  on  a  time  available  basis  and  will  not  seriously 
impede  the  development  of  the  software  for  the  opera¬ 
tional  system. 

To  accomplish  the  modification  of  the  translator 
portion  of  the  compiler  takes  some  time.  In  the 
meantime,  the  developer  can  use  the  nonfunctional 
software  with  the  computer  other  than  the  one  in 
which  it  all  finally  operates.  By  using  a  higher  order 
language  which  is  compatible  to  both  the  old  computer 
and  the  new  one,  actual  code  generation  and  logic 
checks  can  be  made.  When  the  “new”  computer 
equipment  is  ready  the  code  already  generated  can 
be  recompiled  for  the  new  computer.  The  amount  of 


scheduled  linear  time  lost  in  this  effort  is  very  small. 

The  use  of  the  compiler  in  this  way  requires  detailed 
plans.  Only  that  nonfunctional  software  must  be  used 
which  will  be  available  in  the  new  computer.  Well 
defined,  limited  programming  language,  coding  con¬ 
ventions,  and  techniques  are  also  required  In  most 
cases  these  can  be  prepared  in  initial  form  very  early 
in  the  development  cycle. 

Aspects  of  the  approach  described  have  been  suc¬ 
cessfully  used  in  a  recent  application.  The  amount  of 
time  lost  in  the  recompilation  and  in  verifying  the 
new  compiler  was  less  than  one  week.  This  ac¬ 
complishment  was  made  possible  by  careful  pre¬ 
planning  for  the  recompilation  and  the  saving  and 
rerunning  of  recorded  input  and  outputs  from  tests 
made  before  the  recompilation.  The  preplanning 
included  carefully  documented  restrictions  on  the 
use  of  legal  higher  order  statements,  declarations, 
etc.  In  addition  some  portions  of  the  command  sys¬ 
tem  which  were  highly  computer-dependent  were 
deferred  until  the  nonfunctional  software  for  the 
computer  was  available. 

In  summary,  the  recompiling  technique  described 
above  has  been  successfully  used  with  careful  pre¬ 
planning  at  a  considerable  savings  in  schedule  time. 

Circumventing  the  unavailability  of  hardware 

The  unavailability  of  the  computer  itself,  some 
portions  of  it,  or  some  interface  component  may  also 
hinder  the  development  of  software  for  the  command 
system. 

One  way  to  alleviate  the  problem  is  to  simulate 
the  operation  of  the  missing  element  with  the  com¬ 
puter.  This  approach  has  been  used  in  many  instances 
and  has  included  the  simulated  operation  of  the  total 
computer  by  another  computer  as  well  as  the  sim¬ 
ulation  of  the  interfacing  hardware.  Generally,  how¬ 
ever,  the  simulation  programs  must  be  developed  and 
this  additional  work  causes  schedule  delays. 

Other  approaches  are  possible.  One  which  has  been 
used  is  similar  to  that  described  in  the  previous 
section.  Recently  a  particular  configuration  of 
display  equipment  and  computer  was  scheduled  to 
be  available  for  computer  program  development  only 
three  months  before  the  desired  operational  use  of 
the  command  system.  Unless  the  display  portion  of 
the  command  system  was  verified  prior  to  this 
three-month  period,  the  operational  use  date  could 
not  be  met.  There  existed  elsewhere  a  similar  display 
configuration  which  was  mated  with  a  different  com¬ 
puter.  Both  computers  used,  or  were  scheduled  to 
have  available,  the  same  higher  order  language 
(JOVIAL).  The  approach  decided  upon  was  to  code 
the  command  system  in  the  higher  order  language. 
Early  checkout  and  display  verification  was  ae- 
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complished  using  the  available  display  system 
mated  with  the  “wrong”  computer.  When  the  opera¬ 
tional  configuration  became  available  the  computer 
program  was  recompiled  on  the  “right”  computer. 
This  recompiled  computer  program  was  then  used 
w'ith  the  operational  configuration  of  display  equip¬ 
ment  and  the  new  computer  for  continued  verification. 
As  a  result  the  desired  schedule  was  achieved.  The 
technique  described  could  be  employed  to  circumvent 
the  nonavailability  of  other  pieces  of  equipment 
including  the  computer  itself.  This  technique,  how¬ 
ever,  does  not  eliminate  the  need  for  use  of  the 
operational  configured  equipment.  It  only  delays  and 
reduces  the  period  of  time  for  which  it  is  required. 

Again,  the  same  preplanning  precaution  described 
in  the  previous  section  must  be  taken.  The  prob¬ 
ability  of  a  successful  transfer  of  a  high  order  coded 
program  from  one  computer  to  another  computer  is 
significantly  increased  if  the  program  is  constructed 
with  transfer  in  mind. 

Management  for  change  implementation 

The  developer  of  software  for  a  command  system  is 
faced  with  the  knowledge  that  during  the  develop¬ 
ment  process  changes  to  the  software  will  be  re¬ 
quired.  Changes  to  an  in-development  computer 
program  can  affect  both  the  quality  and  the  schedule. 
A  study  of  the  number  of  errors  corrected  in  a  com¬ 
mand  system  during  a  one-month  period  demonstrated 
that  each  time  a  change  was  introduced  the  error  rate 
increased.  A  clear  correspondence  was  found  be¬ 
tween  high  error  rate  and  the  number  of  changes. 
The  problem  that  faces  the  developer  is  how  to 
accommodate  change  without  adversely  affecting 
quality  or  lengthening  the  schedule. 

The  impact  of  a  given  change  varies  not  only  with 
its  size,  complexity,  and  system  involvement  but 
also  as  a  function  of  the  stage  of  development  of 
the  software.  If  a  change  is  identified  early  in  the  de¬ 
sign  phase  it  may  be  possible  to  include  it  in  the 
command  system  with  no  impact  on  quality  or 
schedule.  On  the  other  hand  if  the  change  is  in¬ 
troduced  into  the  system  during  later  stages  of 
testing  it  may  seriously  affect  the  quality  of  the 
problem  or  delay  its  completion  or  both.  In  general 
the  later  a  change  is  introduced  in  the  development 
process  the  greater  its  impact  will  be. 

With  this  rule  of  thumb  and  the  knowledge  that 
change  is  inevitable,  two  courses  of  action  are 
suggested  in  order  to  manage  the  impact  of  change: 


Make  full  provision  in  the  design  of  the  military 
command  system  for  change  implementation,  and 
cut  off  all  changes  during  the  later  phases  of  testing 
and  plan  for  incremental  modification  packages  to 
accommodate  subsequent  changes. 

Some  goals  to  consider  in  designing  a  system  to 
allow  for  changes  include  the  following: 

•  Fractionate  the  system  into  subprograms  so 
that  the  functions  are  as  isolatable  as  possible. 
That  is,  try  to  construct  the  subprogram  so 
that  changes  to  a  function  will  require  changes 
only  to  that  subprogram. 

•  Construct  central  tables  containing  items, 
values,  parameters,  etc.,  used  by  more  than  one 
program.  Thus,  if  a  value  is  changed  the  cor¬ 
rection  need  only  be  made  in  one  place. 

•  Define  with  care  all  communication  items  be¬ 
tween  programs  and  systems.  Document 
these  carefully. 

•  Define  carefully  rules,  conventions  and  pro¬ 
cedures  to  be  used  by  all  programmers  to 
facilitate  and  expedite  understanding.  Prohibit 
involved  and  tricky  programming  techniques. 

•  Insist  that  programmers  provide  documented 
program  comments  which  will  permit  other  pro¬ 
grammers  to  understand  readily  what  is  being 
done. 

•  If  possible,  use  a  higher  order  language  and 
maintain  the  program  in  that  language.  Do  not 
resort  to  corrections  in  binary  code  if  it  can 
be  avoided. 

•  Define  tests  and  results  such  that  if  a  change 
is  made,  it  can  be  easily  verified  that  other 
functions  are  still  operating  properly. 

•  Where  possible,  for  each  parameter  define  a 
range  within  which  the  parameter  can  be  mean¬ 
ingfully  varied. 

The  second  action  suggested  is  to  determine  a 
point  in  time  when  no  other  changes  will  be  accepted, 
and  to  test  thoroughly  this  system  base.  Given  a 
tested  base,  further  changes  can  be  accommodated 
with  increased  confidence  that  they  will  have  a  low 
impact  on  quality  and  schedules.  It  is  generally 
convenient  to  collect  a  set  of  these  later  changes 
and  develop  them  as  a  unit. 

The  combination  of  the  two  courses  of  action  has 
been  successfully  employed  with  improvements  in 
the  quality  of  the  program  and  the  achievement  of 
greater  overall  system  capability  in  shorter  time 
than  would  otherwise  have  been  possible. 
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INTRODUCTION 

Rapid  growth  in  complexity  of  command  and  control 
systems  has  placed  severe  requirements  on  associated 
display  subsystems.  Existing  display  technology  is  gen¬ 
erally  inadequate,  and  it  is  becoming  increasingly  diffi¬ 
cult  to  meet  the  operational  requirements  dictated  by 
present  and  future  systems.  Current  systems  call  for 
displays  that  can  handle  large  volumes  of  data  in  real 
time  and  arc  bright,  multicolor,  of  high  resolution  and 
highly  flexible.  There  arc  no  existing  displays  that  can 
provide  this  total  capability.  New  techniques  must  be 
provided  to  ensure  that  the  display  docs  not  become  the 
weak  link  in  the  information  system  chain. 

In  order  to  provide  continuing  solutions  to  the  display 
problem,  an  extensive  R&D  program  has  been  estab¬ 
lished  in  the  Rome  Air  Development  Center’s  Display 
Branch.  This  program  is  multifold  and  provides  for 
engineering  support  for  systems  now  in  existence  or 
under  development,  engineering  development  of  display 
systems  for  near  future  requirements,  and  research  of 
new  techniques  and  ideas  to  provide  a  basis  for  the  ad¬ 
vanced  displays  of  the  future.  As  a  result  of  this  pro¬ 
gram,  several  display  disciplines  have  been  extensively 
studied  and  show  promise  towards  providing  displays  in 
consonance  with  the  stated  objective  of  the  program. 
This  paper  will  describe  work  under  way  in  the  follow¬ 
ing  two  areas,  laser  displays  and  light  valve  displays. 

The  recent  development  of  the  laser  as  a  practical, 
continuous,  coherent  light  source  has  created  a  new 
display  technology,  that  of  the  laser  beam  display.  In 
concept  form,  one  thinks  of  the  laser  beam  as  being 
analogous  to  the  electron  beam,  however,  some  major 
differences  exist.  While  the  laser  beam  is  an  uncharged 
beam,  thus  requiring  the  development  of  new  tech¬ 
niques  for  its  control,  it  is  not  constrained  to  a  vacuum 
environment  nor  does  it  require  a  special  screen  for 
the  emission  or  control  of  light  as  docs  the  electron 
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beam.  These  advantages  thus  prompted  further  exploita¬ 
tion  of  the  application  of  the  laser  to  displays.  The  most 
general  laser  display  concept  is  shown  in  block  diagram 
form  in  Figure  1 .  A  description  of  the  concept  can  be 
found  under  the  heading  Laser  Display 

Light  valve  displays  offer  several  possible  solutions 
to  the  display  problems  mentioned  above.  Light  valve 
displays  include  all  those  devices  which  use  an  electron 
beam  to  change  an  optical  parameter  of  a  control  layer. 
These  changes,  in  conjunction  with  special  optics,  modu¬ 
late  light  supplied  by  an  external  light  source  in  order 
to  produce  a  display  image.  The  electron  beam  effects 
changes  at  discrete  points  of  the  control  layer.  In  this 
manner  the  image  projected  onto  the  display  screen  is 
an  optical  replica  of  the  charge  image  that  the  electron 
beam  has  deposited.  By  using  a  light  valve  one  retains 
the  flexibility  and  simplicity  of  electron  beam  control 
of  CRT  devices.  In  addition,  one  gains  in  image  size  and 
brightness.  Image  size  is  controlled  by  the  projection 
optics,  while  brightness  is  limited  only  by  the  external 
light  source  and  the  heat  dissipation  capacity  of  the 
control  layer.  A  light  valve  may  be  expected  to  provide 
in  a  large  bright  display,  selective  address  capability, 
high  resolution,  high  writing  speeds  and  controlled 
persistence  available  in  CRT’s.  A  description  of  the 
RADC  light  valve  program  is  contained  under  the  head¬ 
ing  Light  Valve  Displays. 

Laser  display 

In  its  simplest  breakdown,  the  laser  display  consists 
of  three  major  items,  the  laser  source (s),  the  light 
modulators  and  the  light  beam  deflector.  The  laser 
sources  which  have  been  considered  provide  a  con¬ 
tinuous  output  in  the  visible  spectrum  and  are  of  the 
gaseous  variety.  Other  sources,  such  as  semiconductor 
diodes  having  pulsed  outputs  in  the  visible  region  can 
be  considered  when  the  technology  allows.  Pertinent 
characteristics  of  the  laser  sources  as  applied  to  the 
display  problem  are  discussed  later  on. 

A  great  deal  of  research  has  been  expended  on  the 
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Figure  I — Generalized  block  diagram  of  laser  display 


light  modulation  problem  for  various  laser  applications. 
The  statc-of-thc-art  has  advanced  to  the  point  where  a 
DC — 30  MHZ  baseband  modulator  is  practical  thereby 
providing  high  resolution  displays.  The  theory  and  ap¬ 
plication  of  these  devices  arc  also  discussed. 

The  problem  of  deflecting  a  photon  beam  is  con¬ 
siderably  more  difficult  than  that  of  deflecting  a  charged 
electron  beam.  Many  deflection  techniques  are  under 
consideration.  The  electromechanical  resonant  scanner 
is  described  in  detail  since  it  appears  to  be  the  most 
promising  for  achieving  a  high  resolution  raster  display. 
Several  versions  have  been  built*  including  a  525  line 
piezoelectric  device  operating  at  15.75  KH*,  and  a  945 
line  magnetostrietive  device  operating  at  28.35  KH*. 

Finally,  it  is  important  to  consider  the  image  which 
results  from  a  laser  display.  One  major  observation  of 
the  visible  laser  beam  is  a  “sparkling”  sensation  to  the 
viewer  due  to  diffraction  and  interference  effects.  An 
experimental  model  of  a  laser  television  system  has  been 
built  which  assisted  in  making  some  general  observa¬ 
tions  regarding  this  effect.  The  initial  version  provided 
a  525  line  monochromatic  laser  image,  and  a  follow-on 
effort  is  presently  under  way  to  develop  a  model  which 
will  provide  a  525  line  multicolor  laser  image. 


*The  piezoelectric  and  magnetostrietive  scanner  were  both 
developed  by  the  Texas  Inst.  Co.  under  RADC  contracts 
AF30(602 )-327 1  and  AF  30(602)-373I  respectively. 


A.  Pertinent  Characteristics  as  Applied  to  Displays 

The  laser  properties  of  interest  for  the  display  appli¬ 
cation  are: 

•  Spatial  Coherence 

•  Temporal  Coherence 

•  Color 

•  Intensity 

1 .  Spatial  coherence 

Conventional  light  sources  emit  wavefronts  whose 
spatial  phase  characteristics  cannot  be  predicted.  They 
arc  said  to  be  incoherent  sources  of  energy.  In  eontra- 
speet,  radio  waves  and  microwaves  generated  by  man, 
represent  coherent  sources  of  energy.  A  wavefront 
emitted  by  such  oscillators  allows  a  high  degree  of 
phase  correlation  as  a  function  of  time  and  spatial 
variations.  This  fact  makes  possible  the  sophisticated 
design  of  present  day  radar  equipment. 

The  laser  is  a  coherent  source  of  light.  A  gas  laser 
operated  in  the  fundamental  transverse  electromagnetic 
mode,  7'EMoo,  can  be  considered  as  being  a  diffraction 
limited  beam.  Consider  the  optical  system  shown  in 
Figure  2.  The  diffraction  limited  diameter  is  given  by: 

2.4  f  \ 

d=-^r-  (i) 

where: 

d  is  the  diameter  of  the  focused  spot 

f  is  the  focal  length  of  the  lens 

\  is  the  wavelength  of  the  illuminating  source 
D  is  the  diameter  of  the  lens. 
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Figure  2 — Simple  optical  system 


the  two  sources,  i.e.,  the  3  milliwatt  laser  and  the 
1,000  watt  mercury  are  lamp. 

This  property  also  allows  us  to  consider  limited 
aperture  devices  as  practical  for  both  the  modulator  and 
deflector  components.  It  can  be  shown  that  a  reduction 
in  modulator  drive  from  7,000  volts  to  400  volts  can 
be  attributed  to  the  spatial  coherence  property  of  the 
laser. 


If  the  system  is  illuminated  with  laser  energy  of  X 
equal  to  .6328  microns  with  D  equal  to  .127  cm  and 
f  =  3.81  em,  the  minimum  diameter  d  which  can  be 
achieved  is  approximately  46  microns.  The  percentage 
of  incident  energy  within  this  diameter  is  ~  84% 
(Born  &  Wolf,  1959).  Thus,  for  a  3  milliwatt  laser  at 
.6328  microns  (relative  visibility  factor  of  the  eye  for 
this  wavelength  is  .24),  the  flux  within  the  focused  spot 
of  the  above  diameter,  is  approximately  0.4  lumen. 

An  ordinary  light  source  can  be  made  spatially  co¬ 
herent  by  placing  a  very  small  iris  in  the  path  of  the 
beam  at  some  extended  distance  from  the  source,  allow¬ 
ing  only  those  rays  with  the  desired  directionality  to 
pass  through  With  appropriate  collimating  optics,  one 
can  then  make  these  rays  parallel.  The  main  problem 
is  the  large  loss  in  luminous  flux  which  must  be  suf¬ 
fered.  The  flux  (W)  from  an  incoherent  source  in  a 
diffraction  limited  beam  is  given  by  (Hopkins,  1964): 


W  =  BA  o)  (2) 

where; 

B  is  the  radiance  of  the  source 
A  is  the  area  of  the  Airy  disc 
<o  is  the  solid  angle  subtended  at  the  source  by 
the  collecting  lens. 

The  diffraction  limited  diameter  of  the  focused  spot 
was  given  in  Equation  1.  The  area  of  the  Airy  disc  is 
then: 


A  =  n  j1-22  X  fj’  (3) 

The  measure  of  the  solid  angle  (co)  subtended  at  the 
source  by  the  collecting  lens  is  the  area  cut  out  by  the 
cone  from  the  sphere  divided  by  the  square  of  the 
radius  of  the  sphere  or 


11  D- 

W  =  TF 


(4) 


and  combining  the  preceding  equations,  we  obtain  for 
the  flux 


W  = 


(1.22  77  X  r 


B 


(5) 


If  we  consider  a  1,000  watt  mercury  are  which  has  a 
radiance  of  10  W/emVsteradian  in  the  .5461  micron 
line  (at  this  wavelength,  the  conversion  from  watts  to 
lumens  in  620  Iumens/watt,  the  luminous  flux  is  W  — 
.68  X  10'4  lumens.  Therefore,  we  can  see  a  four  order 
of  magnitude  difference  between  the  flux  collected  from 


2.  Temporal  coherence  and  color 

The  temporal  coherence  property  of  the  laser  im¬ 
plies  a  very  narrow  linewidth  or  monochromaticity  of 
the  emitted  radiation.  From  a  display  point  of  view,  the 
narrow  linewidth  means  that  we  can  consider  the  basic 
laser  source  as  a  100  per  cent  pure  or  a  fully  saturated 
color.  By  appropriate  choice  of  the  three  basic  sources 
comprising  a  color  additive  scheme,  it  is  possible  to 
reproduce  a  wider  spectrum  of  colors  than  can  be 
achieved  with  other  color  display  systems.  Using  the 
Hc-Nc  laser  (output  at  .6328  micron)  and  the  Argon 
laser  (outputs  at  .5145  micron  and  .4880  micron) 
to  provide  the  three  primaries,  we  can  plot  the  range 
of  colors  which  may  be  reproduced.  The  color  triangle 
which  results  is  shown  in  Figure  3.  A  comparison  is 
made  to  the  present  day  color  TV  range.  It  is  very  evi¬ 
dent  that  a  higher  frequency  blue  will  be  required  to 
provide  faithful  reproduction  of  the  magenta  and  purple 
portion  of  the  color  spectrum. 


0  LASER  PRIMARIES 
X  COLOR  TV  PRIMARIES 


Figure  3 — Plot  of  3  CW  visible  laser  sources  on  CIE  chart 
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3.  Power  and  intensity 

An  empirically  derived  expression  which  relates  the 
brightness  of  the  display  image  to  the  laser  power  and 
other  system  parameters  is  given  by  Equation  6. 

Bz:C,  G  LPT  (6) 

A 

where 

Lp  —  P  K  C, 

X 

B  is  the  highlight  brightness  of  the  area  under 
consideration  expressed  in  foot  lambcrts; 

C,  is  the  transmission  efficiency  of  the  system 
giving  the  luminous  flux  incident  on  the 
sereen  expressed  as  a  fractional  part  of  the 
total  luminous  flux  available  from  the 
souree; 

G  defines  the  directionality  of  the  screen  relative 
to  that  of  a  lambertian  surface,  expressed 
as  an  integer; 

A  is  the  area  of  the  elemental  area  under  con¬ 
sideration,  or  the  area  of  the  minimum  re¬ 
solvable  element  expressed  in  square  feet; 

L,,  is  the  peak  luminous  flux  available  from  the 
source,  in  lumens. 

T  is  the  duty  cycle  or  the  ratio  of  the  dwell  time 
of  the  laser  beam  on  the  area  under  con¬ 
sideration  expressed  as  a  fractional  part 
of  the  total  available  time  (the  repetition 
rate  of  the  laser  beam  on  any  one  element 
of  the  display  is  chosen  to  be  above  the 
critical  fusion  frequency  of  the  eye,  to 
prevent  a  flickering  image); 

P  is  the  laser  power  in  watts; 

KX  is  the  relative  visibility  factor  of  the  eye  for 
the  particular  wavelength  under  considera¬ 
tion  and 

Ct  is  the  conversion  factor  from  watts  to  lumens 
at  .5550  mierons  and  equals  680  lumens/ 
watt. 

It  is  seen  that  the  brightness  which  ean  be  achieved 
is  a  direct  function  of  the  duty  eycle  as  well  as  the 
laser  power.  This  implies  that  the  method  of  sean  and 
display  format  are  extremely  important  factors  in 
determining  the  brightness  of  the  display.  The  following 
examples  are  chosen  to  demonstrate  this  point. 

Let  us  consider  a  one  watt  Argon  laser  as  the  basic 
souree  of  a  monochromatic  laser  display.  The  average 
visibility  factor  is  0.40  giving  a  luminous  flux  of  272 
lumens  from  the  source.  Then  let  us  impose  the  con¬ 
ditions  that  the  system  has  an  optical  efficiency  of  0.5, 
the  display  image  has  an  area  of  100  square  feet,  the 
display  image  is  composed  of  one  million  resolvable 
elements  and  the  sereen  is  a  lambertian  diffuser  having 
a  gain  of  one.  If  the  defleetor  provides  a  raster  type 


scan,  all  of  the  resolution  elements  are  accessed  in  one 
frame  period  giving  a  duty  eyele  of  10'8.  The  brightness 
which  would  be  achieved  is  1.35  foot  lamberts.  If  the 
defleetor  is  flexible  enough  to  allow  random  or  selective 
accessing  of  the  information  to  be  presented  to  the 
viewer,  then  only  those  resolution  elements  containing 
information  would  have  to  be  accessed  in  one  frame 
period.  For  a  military  display,  this  number  is  on  the 
average  about  2.5  pereent  of  the  total  resoultion 
elements.  Thus  the  duty  eycle  would  be  4  X  10  s,  giving 
a  potential  display  brightness  of  50  foot  lamberts. 

Various  gaseous  lasers  are  available  today  which  can 
provide  cw  visible  output.  Most  notably,  for  the  display 
application,  are  the  He-Ne  laser  and  the  Argon  and 
Krypton  ion  lasers.  Commercial  models  of  the  He-Ne 
laser  can  provide  100  milliwatts  of  power  at  .6328 
mieron.  Commercial  models  of  the  Argon  laser  ean 
provide  5  watts  of  continuous  power  at  dominantly 
.5145  micron  and  .4880  mieron  and  the  Krypton  ion 
laser  can  provide  250  milliwatts  of  cw  power  at  .6470 
micron. 

B.  Electro-optic  (E-O)  modulators 

There  are  various  mechanisms  available  for  effectively 
varying  the  amplitude  of  the  laser  beam.  The  wide 
bandwidth  (DC  to  30  MHZ)  required  to  achieve  a 
high  resolution  display  necessitates  the  use  of  an 
electro-optic  or  similar  high  speed  device.  Two  cate¬ 
gories  will  be  described,  one  which  is  referred  to  as 
the  linear  e-o  effect  and  the  other  as  the  quadratic 
e-o  effect.  The  particular  effect  which  applies  is  a 
function  of  the  symmetry  class  of  the  e-o  crystal. 

1 .  Linear  E-O  effect 

This  effect  is  exhibited  by  crystals  falling  into  the 
42m  crystal  class.  Some  representative  types  are  in  the 
class  of  X  H2  PCX,  where  X  can  be  potassium  or  am¬ 
monium.  This  effeet  has  been  widely  described  in  the 
literature  (Billings,  1949  a,b  1952,  Carpenter,  1950). 
The  classical  mode  of  operation  is  shown  in  Figure  4. 
The  dominant  charaeteristie  of  this  classical  model  is 
that  the  direetion  of  the  applied  eleetrie  field  is  parallel 
to  the  direction  of  propagation  of  the  light  beam. 

Consider  the  schematic  shown  in  Figure  4,  minus 
the  quarter  wave  plate.  The  collimated  input  beam  is 
converted  to  linearly  polarized  light  by  the  first  polar¬ 
izer.  With  zero  electric  field  applied  to  the  e-o  crystal, 
it  is  uniaxial  and  if  the  direetion  of  propagation  of  the 
polarized  light  is  parallel  to  the  optic  axis  of  the 
crystal,  the  light  propagates  undisturbed  through  the 
crystal.  Since  the  analyzer  and  polarizer  are  oriented 
with  their  preferred  directions  orthogonal  with  respect  to 
eaeh  other,  none  of  the  light  is  transmitted  through  the 
analyzer.  When  an  eleetrie  field  is  applied  along  the  z 


I 
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PREFERRED  DIRECTION 
OF  POLARIZATION 


0°  Z-CUT 


\ 


Figure  4 — Longitudinal  PockeFs  effect  modulator  (electric 
field  parallel  to  direction  of  propagation  of  light) 


direction  of  the  e-o  crystal,  it  becomes  biaxial.  The 
index  of  refraction  is  different  along  the  two  axes  and 
the  difference  is  given  by: 

A  "  =  -  V-  (?) 

where 

An  is  the  change  in  the  index  of  refraction 
X  is  the  wavelength  of  the  incident  light 
r  is  the  retardation  expressed  as  an  integer 
d  is  the  crystal  thickness  in  the  Z  direction 

and 


where 

r63  is  electro-optic  coefficient  in  the  case  where 
the  electric  field  is  applied  along  the  Z 
direction,  meters/ volt 
Vx  is  voltage  in  the  Z  direction,  volts. 

When  the  voltage  applied  to  the  crystal  is  sufficiently 
large  to  produce  a  retardation  of  r=  V2,  all  of  the 
light  is  transmitted  through  the  analyzer.  In  the  model 
shown,  a  nominal  voltage  required  to  produce  the  half¬ 
wave  retardation  is  7,000  volts  for  KDP. 

The  primary  function  of  the  quarter  wave  plate  is 


to  provide  for  operation  along  the  linear  portion  of  the 
modulation  transfer  curve  and  is  not  discussed  here. 

It  is  possible  to  design  a  linear  e-o  modulator  which 
takes  advantage  of  a  long  transmission  path  compared 
to  aperture  dimension  (Baker,  1965).  A  schematic 
diagram  is  shown  in  Figure  5.  Here  the  electric  field 
and  direction  of  propagation  of  the  light  beam  are 
orthogonal,  and  the  device  is  referred  to  as  a  transverse 
modulator.  The  total  phase  shift  for  an  incident  light 


analyzer 


Figure  5 — Linear  F-O  PockeFs  effect  (electric  field  direction 
orthogonal  to  direction  of  propagation  of  light) 
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beam  polarized  at  45  to  the  y  axis  of  crystal  A  or  x 
axis  of  crystal  B  is  given  as  follows  (Wenz,  1964): 

2  n  l 

A  (/>  =  </>,  —  <j>y=  — — [(nx  +  n*)  —  (ny  +  nr)] 


or 

A  2  n  /  x  2  n  / 

A  <f>  =— (nx  —  ny)  =  A  n 


(9) 


(10) 


For  equation  10  to  be  valid,  the  lengths  of  crystals 
A  and  B  must  be  held  equal  to  within  .0001".  Sub¬ 
stituting  for  A  n  where: 

r63  n03  Vr 


A  n  = 


we  obtain 


A  </>  = 


2  n  /,  Tea  n03  V, 


(11) 


(12) 


X  d 

which  shows  the  dependence  of  the  retardation  on  the 
// d  ratio. 


Some  typical  crystal  parameter  and  geometry  values 
for  this  type  modulator  are  as  follows: 

rM  =  5.25  X  10r’  mierons/volt  (KDP) 
n0  =  1.468 
X  =  0.6328  microns 
/  =  7.62  cm. 
d  =i  .  1  8  cm. 

and  to  achieve  100  per  cent  modulation,  the  half  wave 
voltage  required  is  approximately  440  volts.  This  com¬ 
pares  favorably  to  the  7  KV  for  the  longitudinal  mod¬ 
ulator.  A  photograph  of  a  typical  c-o  modulator  is 
shown  in  Figure  6. 


2.  Quadratic  E-O  effect 

Many  isotropic  materials  when  placed  in  an  electric 
field  behave  like  a  uniaxial  crystal  with  the  optic  axis 
in  the  direction  of  the  field.  In  this  case,  the  induced 
birefringence  is  a  function  of  the  square  of  the  applied 
electric  field  and  the  phenomenon  is  called  the  Kerr 
effect  or  the  quadratic  e-o  effect. 

The  Kerr  effect  in  liquids  is  a  well-known  phenom¬ 
enon  and  is  discussed  in  many  textbooks.  A  commonly 
used  liquid  is  nitrobenzene  and  for  a  cell  having  a 
2.5  mm  aperture  (electrode  spacing)  and  a  2  cm. 
length,  the  nominal  voltage  required  for  100  percent 
modulation  is  6,500  volts.  This  value  is  large  in  com¬ 
parison  to  the  440  volts  required  of  the  modulator  pre¬ 
viously  described. 

However,  there  exists  a  unique  class  of  crystalline 
elements,  of  the  Pcrovskitc  family,  which  in  their  para- 
eleetrie  phase  exhibit  a  strong  quadratic  effect.  The 
dependence  of  the  applied  field  as  a  function  of  the 
crystal  properties  and  as  a  function  of  the  number  of  n 
phase  retardations  has  been  shown  to  be  (Gcusic,  1963, 
1964) 


Figure  6 — E-O  modulator 


v  >  =  4  _*_*! _ 

(mil)  [n„3  (g„  -  g„)  e7 

where 


(13) 


V  is  the  voltage  applied  to  the  crystal  (volts) 
m  is  the  integral  number  of  the  FI  phase  retarda¬ 
tions 

X  is  the  vacuum  wavelength  of  light  (meters) 
(gn  -  gn)  is  the  quadratic  c-o  coefficient  m4/ca 
m4/c2 

d  is  electrode  spacing  (meters) 

/  is  crystal  length  in  direction  of  light  propaga¬ 
tion  (meters) 

€  is  static  dielectric  constant,  farads/meters  and 
n„  is  index  of  refraction  in  absence  of  electric 
field. 

Equation  13  is  valid  when  the  electric  field  is  applied 
along  a  (001)  direction  and  the  light  propagates  in 
the  (100)  direction. 

Reported  data  for  KT.  Nb  O  (KTN) 

r  .65  .35  3 
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which  has  a  Curie  Temperature  of  ^  283 °K  is  listed  in 
Table  I. 


Table  I — Properties  of  KIN  for  X  —  .6328  micron 
and  T  -  295°  K 
Parameter  KTN 


g.i  -gi-- 
n„ 

0 

€ 

I\  (saturation 
polarization) 


0.174  mVc2 
2.29 
^  150 
10,000 
10  /x  C/cm2 


Since  the  effect  is  quadratic,  a  dc  bias  can  significant¬ 
ly  reduce  the  differential  half-wave  voltage  required  for 
100  percent  modulation.  This  can  be  readily  seen  by 
looking  at  the  following  example.  Let  us  choose  a  block 
of  KTN  having  the  dimensions  d  2mm  and  1  =  10 
mm.  The  voltage  required  to  achieve  a  phase  retarda¬ 
tion  of  TT,  2  TT,  3  IT,  etc.  is  shown  in  Table  II.  For 
100  percent  modulation,  it  is  only  required  to  achieve 
a  differential  phase  shift  of  II  radians.  Thus,  if  wc 
apply  a  dc  bias  of  330  volts  to  the  crystal  described 
above,  the  required  ac  peak  drive  is  22  volts.  This  will 
allow  us  to  go  from  a  full  “off”  to  fully  “on”  condition 
of  the  modulator.  A  comparison  of  the  KDP  transverse 
modulator  and  the  KTN  modulator  is  shown  in  Table 
III. 


Table  II — Voltage  vs.  phase  retardation  in  KTN 


Phase 


Vollnge  Differential  Voltage  for  II 


Retardation 

(volls)  Phase 

Retardation  (volts) 

II 

124 

2  II 

176 

52 

3  11 

215 

39 

4  11 

248 

33 

5  11 

278 

30 

6  11 

305 

27 

7  11 

330 

25 

8  11 

352 

22 

Table  III — Comparison  of  KDP  transverse  modulator 

and 

KTN  modulator 

KTN 

KDP 

Geometry 

2mm  X  10mm 

2mm  X  77mm 

Drive  Voltage 

22  volts 

440  volts 

DC  Bias 

330  volts 

Dissipated 

Power 

.54  watts 

.47  watts 

Required 

Power 

40  watts 

46  watts 

It  is  possible  to  achieve  a  baseband  modulation  band- 
•  .dth  of  30  MH,  with  either  of  the  transverse  modu¬ 
lators  described. 

C.  Laser  deflector 

There  arc  various  known  phenomena  for  deflection 
of  light  beams.  The  unique  properties  of  the  laser 
allow  certain  techniques  to  be  considered  and  others 
to  become  practical  that  normally  could  not  be  con¬ 
sidered  or  were  not  practical  with  ordinary  light  sources. 
A  categorization  of  the  various  techniques  applicable 
to  displays  and  the  state  of  development  is  shown  in 
Table  IV.  Two  of  the  more  promising  techniques  for 
raster  displays  are  the  resonant  piezoelectric,  and  mag- 
netostrietivc  type  scanners.  Techniques  applicable  to  a 
random  or  selective  type  accessing  format  arc  in  the 
research  stage  and  are  not  described  here.* 

1 .  Resonant  piezoelectric  scanner 

A  unique  electro-mechanical  scanner  has  been  de¬ 
veloped  for  specific  laser  display  application  by  the 
Texas  Instruments  Co.  under  RADC  contract  AF30- 
f 602 ) -327 1 .  The  principle  of  operation  is  as  follows. 
A  resonant  piezoelectric  driven  fiber  and  mirror  com¬ 
bination  produce  a  circular  scan  of  light  at  the  horizon¬ 
tal  repetition  frequency  for  the  raster  line  standard 
desired.  The  circular  scan  is  converted  to  a  linear  scan 
by  means  of  a  fiber  optic  assembly.  The  vertical  scan, 
60  H*  repetition  frequency,  is  generated  by  a  galvan- 
ometrie  driven  oscillating  mirror.  The  primary  reason 
for  first  generating  a  circular  scan  is  that  this  allows  the 
piezoelectric  cartridge  to  be  operated  at  resonance  and 
thus  a  large  angular  deflection  is  achieved,  of  the  order 
of  12  degrees.  With  this  magnitude  of  deflection  and 
considering  the  diffraction  limited  condition,  the  scan¬ 
ner  has  the  capability  of  resolving  1,000  horizontal 
elements.  A  schematic  diagram  of  the  piezoelectric 
scanner  is  shown  in  Figure  7.  The  various  constituents 


PIEZOELECTRIC  SCANNER  ASSEMBLY 


INCIDENT  COLLIMATED 
LASER  SEAM 


FOCUSSED  LASER  BEAM 

Figure  7a, b — Mirror  position — unexcited  state 


*£>leetive  access  laser  display  beam  positioner  study  wilh 
Hughes  Aircraft  Co.,  Malibu,  Calif  AF30(602 )-4097. 
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Category 

Electro-Mechanical 


Refraction 


Diffraction 


Coherent  Scanner 


Table  IV — Laser  beam  deflection  techniques 

State  of  Development 

Specific  Type 


(Outstanding  Characteristic) 


Rotating  Drum  Scanner  Operational  (limited  to  raster  scan  at  repetition  fre¬ 

quencies  of  10  to  15  KHz) 

Resonant  iPezoelectrie  Scanner  Operational  (can  provide  raster  scan  at  repetition  fre¬ 
quencies  of  15.75  KHz  [525  line]  and  21.87  KHz  [729 
line]) 


Resonant  Magnetostrietive 
Scanner 


Operational  (can  provide  raster  scan  at  repetition  fre¬ 
quencies  of  28.35  KHz  [945  line]  and  30.87  KHz  [1029 
line]) 


Electro-Optic  Prism 


Research  stage  (materials  limited  and  resolution 
limited) 


Ultrasonic  Cell 
Debye-Sears  Effect 
Bragg  Angle  Incidence 

Coherent  Optical  Phased  Array 


Research  stage  (resolution  limited) 

Research  stage  (transmission  limited) 

Application  stage  (characteristics  not  completely  deter¬ 
mined,  however  appears  promising  for  a  raster  scan) 

Research  stage  (requires  high  degree  of  stability  and 
is  resolution  limited) 


Binary  Electro-optic  Convergent  Beam 

Deflector 

Wollaston  Prism  Type 
Multidirectional  Laser  Various  types 


Research  stage  (main  problems  are  materials  limita¬ 
tion  and  low  switching  speed) 

Research  stage  (mainly  in  idea  stage) 

Idea  and  early  research  stage  (no  prediction) 


of  the  scanner  are  shown  in  Figure  7a.  Bonding  of  the 
interfaces  is  accomplished  with  an  epoxy  cement.  The 
position  of  the  focussed  laser  beam,  with  no  excitation 
applied  to  the  scanner  is  shown  in  Figure  7b.  In  Figure 
7c,  one  of  the  many  possible  positions  of  the  focussed 
laser  beam  with  the  piezoelectric  cartridge  excited,  is 
depicted.  The  electrical  connections  of  the  piezoelec¬ 
tric  cartridge  are  shown  in  Figure  7d. 

PZT-5 


LASER  BEAM 


INC  I0ENT  COLLIMATED 
LASER  BEAM 


MIRROR  POSITION  -  ONE  OF  MANY  EXCITED  STATES 


TWO  SINE  WAVE 
GENERATORS,  FREQUENCY 
90*  OUT  OF  PHASE 


END  VIEW  OF  CARTRIDGE  -  ELECTRICAL  CONNECTIONS 

Figure  7c,d — Piezoelectric  scanner,  conceplual 


A  photograph  of  the  piezoelectric  scanner  is  shown 
in  Figure  8.  A  photograph  of  the  fiber  optic  circle  to 
line  converter  is  shown  in  Figure  9. 


Figure  8 — Piezoelectric  scanner 
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Figure  9 — Fiber  optic  circle  to  line  converter 

The  motion  of  the  fiber  is  governed  by  the  equation 
of  motion  of  a  flexural  wave  in  a  beam.  Detailed  dis¬ 
cussion  of  its  operation  lias  been  previously  reported 
(Baker,  C.  E.,  1965).  Some  typical  dimensions  of  a 
scanner  built  for  operation  at  15.75  KHz  arc: 

•  Clevite  PZT-5  Cartridge,  60099 

•  .005"  diameter  fused  silica  fiber  having  a  nomi¬ 
nal  length  of  .  1 50" 

•  Spherical  mirror  having  a  diameter  of  .050"  and 
thickness  of  .003".  Nominal  focal  length  is 
1.5". 

•  For  a  12°  deflection,  the  two  sinusoidal  signals 
required  are  nominally  100  volts  peak-to-peak 

A  photograph  of  the  vertical  scanner  is  shown  in 
Figure  10.  The  significant  achievement  is  that  a  linear 
scan  was  obtained  with  a  1  millisecond  flyback  time. 
A  total  deflection  of  10  degrees  was  used  from  this 
scanner,  although  large  deflections  are  possible. 


Figure  10 — Vertical  scanner 


2.  Resonant  magnetostrictive  scanner 

Various  problems,  the  most  significant  being  life¬ 
time,  were  encountered  with  the  piezoelectric  resonant 
scanner  at  higher  repetition  frequencies,  i.e.,  28.35  KHz. 
Thus  a  research  program  was  initiated  to  explore  al¬ 
ternate  techniques  for  providing  a  high  resolution  de¬ 
flector.  This  work  was  performed  by  the  Texas  In¬ 
struments  Co.,  under  RADC  contract  AF30(602)- 
3731.  The  magnetostrictive  vibrator  appears  to  be 
highly  suited  for  this  application.  A  photograph  of  a 
typical  magnetostrictive  scanner  is  shown  in  Figure  11. 
The  basic  operation  is  similar  to  that  of  the  piezoelectric 
scanner  in  that  it  provides  a  resonant  circular  scan. 
The  mode  of  operation  is  the  torsional  mode,  requiring 
two  independent  scanners  to  generate  the  circular  lis- 
sajous  figures.  The  magnetostrictive  vibration  is  ampli¬ 
fied  by  an  amplitude  transformer  which  can  be  ex¬ 
ponential,  fourier  or  stepped  (Crawford,  A.,  1955). 
Typical  deflections  of  18°  have  been  obtained  at 
30KHz  with  a  reflective  mirror  having  a  diameter  of 
.38  cm.  This  particular  scanner  appears  most  promis¬ 
ing  for  obtaining  a  high  resolution  laser  display  of  the 
raster  type. 


Figure  1 1 — Magnetostrictive  scanner 


I).  Experimental  laser  display,  monochromatic 
model 

An  experimental  model  of  a  laser  display  was  con- 
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structed  to  demonstrate  the  feasibility  of  this  type  dis¬ 
play.  This  model  was  designed  and  fabricated  by  the 
Texas  Instrument  Co.,  under  RADC  contract  AF- 
30(602)3271.  The  laser  source  used  in  the  model  was  a 
50  milliwatt  He-Ne  laser  operating  at  a  wavelength  of 
.6328  microns.  A  photograph  of  the  model  is  shown 
in  Figure  12.  The  unit  was  designed  to  accept  either 
of  KTN  having  the  dimensions  d  =  2mm  and  /  =:  10 
an  r-f  television  signal  (off-the-air  broadcast)  or  a  1.5 
volt  composite  video  closed  circuit  signal. 


Figure  12 — Laser  display  model,  monochromatic 


A  functional  block  diagram  of  the  model  is  shown 
in  Figure  13  and  the  relationship  of  the  various  optical 
components  in  Figure  14. 

The  uniphase  wavefront  from  the  laser  is  amplitude 
modulated  by  a  potassium  dihydrogen  phosphate 
(KDP)  electro-optic  modulator  of  the  transverse  type 
previously  described.  The  bandwidth  of  the  clectro- 
optic  modulator  was  5MHz,  sufficient  to  achieve  nor¬ 
mal  broadcast  television  resolution. 

The  modulated  light  beam  is  scanned  into  a  525  line 
television  raster  by  a  piezoelectric  resonant  scanner  de¬ 
flection  scheme.  A  5 Vi  inch  focal  length,  f/2  aperture 
projection  lens  is  used  to  produce  a  40  inch  square 
image  with  a  25  foot  projection  distance.  A  photograph 
of  the  image  thus  generated,  is  shown  in  Figure  15. 
The  quality  of  this  image  is  affected  by  numerous  fac¬ 
tors,  of  which  alignment  of  the  subcomponents  in  the 
optical  train  is  a  major  factor.  The  performance  which 
was  achieved  is  described  in  Table  V. 

E.  Multicolor  laser  display  model 

The  ultimate  objective  in  the  laser  display  program 
is  to  develop  the  capability  for  generating  multicolor 
images.  In  military'  command  and  control  displays  of 
the  alphanumeric  variety,  the  desired  colors  arc  red, 
green,  blue,  magenta,  cyan,  yellow  and  white.  The  par¬ 
ticular  attractiveness  of  the  laser  display  from  a  color 
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Figure  13 — Functional  block  diagram,  laser  display 
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Figure  14  -Laser  display  optical  train 


Table  V — -Laser  display  model  (monochromatic) 


Color 
Resolution 
Contrast 
Image  Size 
Brightness 

Transmission 
Efficiency 
Laser  Power 


performance 
Red  and  Black 

Equivalent  to  525  line  TV  system 
45:1  (measured  in  dark  environment 
40"  X  40"  at  25  ft.  projection  distance 
3  ft-kunberts  with  a  highly  directional 
screen  (gain  =  32) 

1 5 c/(  (major  loss  in  fiber  optic  scan 
converter) 

50  nnv  (o'  .6328  mieron 


point  of  view  is  that  the  color  mixing  can  be  accom¬ 
plished  prior  to  the  deflector  unit,  thus  precluding 
misregistration  at  the  screen.  Thus  an  effort  was  ini¬ 
tiated  to  look  at  the  possibility  of  combining  the  out¬ 
puts  of  several  lasers  to  generate  a  multicolor  image.* 
The  optical  layout  of  the  multicolor  model  being 
fabricated  is  shown  in  Figure  16.  The  three  primary 
spectral  components  are  derived  as  follows:  the  red 
from  a  He-Ne  laser  operating  at  .6328  mieron  and 
the  blue  and  green  from  an  Argon  laser  operating  at 
.4880  micron  and  .5145  micron  respectively.  The 
three  beams  are  then  individually  modulated  and  com¬ 
bined  by  means  of  a  mixing  prism.  The  combined  beam 
then  varies  in  hue,  saturation  and  intensity  in  corre¬ 


*This  work  is  being  performed  by  the  Texas  Instruments  Co., 
under  R  ADC  contract  AF.'t0(602  )-39 10. 


spondence  to  the  modulation  input  to  the  three  light 
modulators.  The  combined  beam  is  then  scanned  and 
projected  in  the  same  manner  as  accomplished  in  the 
525  line  monochromatic  model. 

At  this  time,  work  is  continuing  in  the  fabrication  of 
the  deliverable  unit.  A  photograph  of  the  unit  is  shown 
in  Figure  17.  A  commercial  color  television  receiver  is 
used  as  the  source  of  the  video  signals  that  modulate 
the  three  light  beams  and  can  be  seen  in  the  photograph. 
At  this  early  stage  of  development  some  preliminary 
multicolor  images  have  been  generated;  however, 
meaningful  photos  are  not  available  at  this  time.  The 
completed  unit  was  due  for  delivery  at  RADC  in 
October  1966.  It  is  emphasized  that  this  model  is  de¬ 
veloped  for  the  sole  purpose  of  looking  at  the  problems 
associated  with  color  generation  using  the  laser  dis¬ 
play  concept  and  is  not  for  field  use. 

F.  Laser  display  system  considerations 

Previous  discussion  has  centered  upon  the  principles 
and  characteristics  of  the  components  required  to  gen¬ 
erate  a  coherent  light  image.  It  is  of  interest  to  deter¬ 
mine  areas  of  uniqueness  of  this  image  and  their  ef¬ 
fects  upon  the  observer.  At  this  time,  the  only  area  of 
major  difference  which  is  seen  is  that  of  the  granularity 
associated  with  a  coherent  light  image. 

This  phenomenon  was  first  observed  in  the  scattered 
light  from  a  Hc-Ne  gas  laser  emitting  in  the  visible 
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Figure  15 — Laser  display  image,  monochromatic 


portion  of  the  spectrum  at  .6328  microns.  Various  ex¬ 
planations  have  been  given,  all  in  good  general  agree¬ 
ment  (Rigdcn,  J.  D.,  1962).  A  scintillation  associated 
with  the  granular  pattern  due  to  relative  motion  be¬ 
tween  the  reflective  surface  and  the  observer  has  given 
rise  to  the  term  “sparkle”  associated  with  this  phenom¬ 
enon.  The  basic  observed  characteristics  of  this  oc¬ 
currence  arc: 

a.  An  appearance  of  bright  and  dark  areas  in  the 
laser  light  which  is  scattered  from  a  matte  surface. 
The  size  of  the  granules  (areas)  arc  a  function  of  the 
viewing  distance  and  angular  aperture  of  the  optical 
system. 

b.  A  motion  of  the  pattern  as  a  function  of  observer 
motion,  the  direction  of  the  motion  differing  for  dif¬ 
ferent  observers. 

c.  A  disappearance  of  the  granular  pattern  by  ran¬ 
dom  motion  of  the  screen  or  use  of  a  colloidal  solu¬ 


tion,  i.c.,  milk,  as  the  reflecting  element. 

The  explanation  of  this  occurrence  lies  in  the  fact 
that  the  coherent  light  scattered  by  the  diffused  surface 
forms  a  complex  diffraction  pattern.  The  granularity 
arises  from  interference  of  the  diffraction  pattern.  The 
scintillation  arises  from  the  random  motion  of  the 
observer,  or  his  inability  to  remain  perfectly  stationary. 
The  motion  of  the  pattern  with  observer  motion  is  a 
result  of  parallax.  The  differences  of  the  direction  of 
motion  observed  by  different  observers  is  a  function 
of  the  focus  plane  of  the  eye,  or  for  example,  the 
nearsightedness  or  farsightedness  of  the  observer.  The 
granularity  is  not  observed  when  the  laser  light  is  re¬ 
flected  from  a  colloidal  solution,  i.e.,  milk,  due  to  the 
large  Brownian  motion  of  the  molecules  in  solution. 
This  effectively  causes  an  averaging  out  of  the  inter¬ 
ference  effect  as  for  an  ordinary  incoherent  light  source. 

This  occurrence  has  also  been  observed  with  a  dif- 
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Figure  16 — Optical  layout,  multicolor  model 


F'igure  17 — Laser  display  model,  multicolor  unit 

fraction  limited  beam  of  unfiltered  sunlight  and  for 
light  from  a  high  pressure  Hg  are,  supporting  the 


hypothesis  that  it  is  a  function  of  the  spatial  eoherenec 
of  the  source  rather  than  its  spectral  purity. 

The  occurrence  or  nonoccurrence  of  this  phenomenon 
in  the  laser  display  image  is  a  function  of  the  method 
by  which  the  image  is  generated.  Observations  have 
been  made  for  several  different  types  of  deflection 
sehemes.  A  two-dimensional  image  was  first  generated 
using  two  piezoelectric  driven  mirror  scanners  to  pro¬ 
duce  a  lissajous  pattern.  A  granularity  of  the  image 
was  observed.  Also,  a  laser  television  system  was  built 
(described  in  the  previous  section)  and  a  2-D  pictorial 
image  was  thus  generated.  The  image  did  not  pave  the 
characteristic  granularity.  In  the  latter  deflection  tech¬ 
niques,  a  fiber  optic  scan  converter  is  used  following 
the  horizontal  scanner  to  convert  a  circular  sean  to  a 
linear  scan.  It  is  this  element  which  effectively  destroys 
the  spatial  coherence  of  the  laser  beam  and  in  effect 
the  sparkle  pattern.  A  laser  television  image  generated 
by  a  deflection  scheme  consisting  of  a  nonmeehanieal 
technique  for  the  horizontal  scan  and  a  vertically  os- 
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dilating  mirror  to  produce  the  vertical  scan  was  re¬ 
ported  to  exhibit  the  granularity.  Thus  the  only  con¬ 
clusions  which  can  be  made  is  that  this  occurrence 
will  depend  upon  the  extent  of  the  diffraction  limit  of 
the  beam  which  is  imaged  on  the  reflecting  surface, 
and  that  methods  arc  available  for  negating  the  effect. 

A  limited  subjective  impression  can  be  stated  for  the 
effect  of  this  granularity  on  the  observer.  Many  ob¬ 
servers  have  indicated  that  their  ability  to  concentrate 
on  the  image  which  possesses  this  “sparkle”  is  con¬ 
siderably  impaired.  It  is  probable  that  it  could  have  the 
same  effect  as  a  flickering  image  on  the  observer.  How¬ 
ever,  qualitative  conclusions  can  only  be  drawn  after 
extensive  experimentation  has  been  performed. 


SUMMARY 


The  present  work  under  way  at  RADC  in  the  laser 
display  area  has  been  described.  It  can  be  seen  that 
the  work  is  experimental  and  has  attempted  to  obtain 
a  good  understanding  of  the  nature  of  the  laser  dis¬ 
play.  Recent  developments  in  the  laser  area  indicate 
that  sufficient  power  will  be  available  shortly  to  gen¬ 
erate  a  large  screen,  high  brightness  laser  display. 
However,  the  life  and  reliability  of  such  a  display  re¬ 
quires  more  experience  with  the  major  components  to 
be  used.  Laser  efficiency  is  poor  at  the  present  time 
and  has  the  effect  of  decreasing  the  lifetime  of  the  as¬ 
sociated  plasma  tube.  The  effect  of  the  high  power  (of 
the  order  of  5  watts)  on  the  optical  components  would 
have  to  be  determined. 

At  any  rate,  it  appears  that  one  can  predict  a  set 
of  characteristics  which  could  possibly  be  realized  in 
the  not-too-distant  future.  These  are  shown  in  Table 
VI.  It  must  be  kept  in  mind  that  numerous  trade-offs 
arc  possible  amongst  the  many  parameters  and  values 
listed. 


Tabic  VI — Laser  Display  Goals 
Image  Size  Up  to  100  sq.  ft. 

7  color  capability:  red,  green,  blue,  yellow, 
cyan,  magenta  and  white. 

50  ft-lamberts 

Will  be  governed  by  specific  system  con¬ 
straints  Screen  gains  higher  than  3  com¬ 
pared  to  a  perfect  lambertian  diffuser  prob¬ 
ably  will  not  be  useful. 

Tactical  Display — accuracy  of  1  part  per 
1 000. 

Pictorial  Display — Deviation  from  perfect 
linearity  can  be  as  high  as  5%. 

33  milliseconds 

Raster  Display:  945  vertical  lines 

1000  horizontal  lines 

Random  Access  Display:  1024  ^  1024  re¬ 
solvable  elements. 

64  symbol  capacity  with  rapid  change 
capability. 

Contrast  Ratio  100  :  I  as  measured  in  a  dark  environment. 


Color 

Brightness 
Screen  Gain 


Linearity 

Requirements 


Update  Time 
Resolution 


Symbology 


Light  valve  displays 

A.  Eidophor  large  screen  television  projector 

The  only  commercially  available  light  valve  today 
is  the  Eidophor  Large  Screen  Television  Projector, 
a  product  of  Grctag  Ltd.,  Regensdorf,  Switzerland.  It 
is  available  in  several  line  standards,  in  black  and  white, 
sequential  or  simultaneous  color  models.  The  unit  de¬ 
scribed  here  is  the  1,000  line  simulataneous  color  pro¬ 
jector.  The  basic  operating  principles  of  the  device 
and  certain  characteristics  will  also  apply  to  other 
Eidophor  models. 

Figure  18  is  a  conceptual  drawing  of  the  Eidophor 
Light  Valve.  The  2,500  watt  xenon  light  source  is 
focussed  onto  a  cold  mirror  for  infrared  removal  and 
uniformly  illuminates  the  picture  window.  The  image 
of  the  picture  window  is  projected  onto  the  spherical 
mirror  via  the  Schliercn  bar  system.  For  simplicity 
only  three  of  the  actual  six  horizontal  Schlicren  bars 
are  shown.  The  bar  system  and  spherical  mirrors  arc 
so  positioned  that  light  incident  on  the  spherical  mir¬ 
ror  is  reflected  to  the  bar  system  and  back  to  the 
light  source. 

To  illuminate  the  screen,  part  of  the  light  returning 
to  the  bar  system  from  the  spherical  mirror  must  be 
slightly  deflected  to  pass  between  the  bars.  The  media 
employed  for  this  purpose  is  a  0.1  mm  thick,  trans¬ 
parent,  viscous  oil  film  spread  evenly  over  the  spherical 
mirror.  Provided  that  the  control  layer  is  smooth,  the 
light  from  the  mirror  is  undcflcctcd  and  the  screen 
remains  dark.  If  the  oil  surface  is  deformed  at  a  point, 
a  portion  of  the  light  illuminating  that  point  is  deflected 
and  passes  between  the  bar  system  producing  a  cor¬ 
responding  point  of  light  in  the  dark  field  at  the  screen. 
The  amount  of  light  that  reaches  the  screen  is  propor¬ 
tional  to  the  slope  and  depth  of  the  oil  deformation 
(sec  detail  upper  right.  Figure  18). 

The  means  used  to  deform  the  oil  are  electrostatic 
forces.  The  metallized  surface  of  the  spherical  mirror 
and  the  surface  of  the  oil  layer  can  be  visualized  as 
a  capacitor.  When  electric  charges  are  placed  on  the 
oil  surface,  attractive  forces  appear  between  the  two 
surfaces.  The  oil  is  basically  a  dielectric  so  a  charge 
applied  at  a  point  will  remain  concentrated  and  exert 
localized  pressure  proportional  to  its  charge  density. 
An  electron  gun  directly  deposits  charge  on  the  oil 
layer. 

The  electron  beam  scans  the  oil  in  raster  format. 
When  the  electron  beam  spot  size  is  large  enough  so 
that  adjacent  lines  on  the  oil  layer  touch  each  other, 
the  charge  will  be  evenly  distributed  over  the  entire 
frame.  T  he  oil  in  the  frame  area  is  subjected  to  con¬ 
stant  pressure  and  remains  smooth  and  the  screen 
appears  dark. 


Real  Time  Display  Techniques  233 


Figure  18 — The  eidophor  light  valve  concept 


Foeusing  the  electron  beam  to  a  smaller  size,  so  that 
adjacent  lines  do  not  touch,  establishes  a  non-uniform 
charge  distribution.  The  pressure  distributions  will  like¬ 
wise  be  non-uniform  and  will  cause  deformation  in  the 
oil  layer.  By  modulating  the  electron  beam  focus  with 
a  video  signal  and  scanning  the  electron  beam,  a  video 
image  can  be  produced  at  the  screen. 

A  television  frame  is  written  every  1  /30th  second. 
If  one  raster  is  not  to  impair  the  succeeding  one,  the  oil 
layer  must  be  smoothed  again  before  the  new  frame  is 
begun.  Two  things  are  required  for  this  to  occur:  the 
deposited  charge  must  be  removed  and  the  surface  ten¬ 
sion  must  remove  the  deformation.  Conductive  additives 
are  included  in  the  oil  to  facilitate  charge  removal  in 
1  /50th  second.  The  time  required  for  the  surface  tension 
to  smooth  the  deformations  depends  upon  the  viscosity 
of  the  oil.  The  proper  viscosity  is  obtained  by  control¬ 
ling  the  temperature  of  the  oil  (Gretag  Ltd.,  undated). 


To  obtain  full  color,  the  Simultaneous  Color  Projec¬ 
tor  utilizes  dichroic  filters  to  break  up  the  light  from 
the  light  source  into  the  three  primary  colors:  red, 
green  and  blue.  These  individual  light  beams  are  then 
directed  to  essentially  identical  cassettes,  as  described 
above.  Each  electron  gun  is  provided  with  the  proper 
video  modulation  for  its  color.  The  images  are  pro¬ 
jected  to  the  screen  where  they  are  super-imposed  in  a 
full  color  picture. 

The  present  capabilities  of  the  Eidophor  are  tabulated 
below: 

•  Resolution:  up  to  1000  lines 

•  Luminous  Flux:  4000  lumens 

•  Uniformity  of  Field:  Approximately  30%  fall-off 
center  to  edge 

•  Contrast  Ratio:  100:1 

•  Linearity:  2% 

•  Optical  Efficiency:  5% 
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The  Eidophor  is  a  complete  projector  requiring  only 
power  and  standard  TV  signal  inputs.  All  electronics, 
optics,  correction  circuitry,  etc.  are  included  in  the 
projector. 

For  command  and  control  military  applications  the 
Eidophor  as  presently  manufactured  has  certain  unde¬ 
sirable  features.  The  fact  that  raster  scanning  is  em¬ 
ployed  to  help  smooth  the  oil  layer  means  that  com¬ 
puter  generated  data  must  first  be  converted  by  com¬ 
plex  buffering  equipment  before  the  data  is  in  proper 
form  for  input  to  the  Eidophor.  Random  scan  is  the 
more  desirable  mode  of  operation  for  computer  driven 
displays.  RADC  is  currently  investigating  the  possibility 
of  using  the  Eidophor  in  a  random  scan  mode.  Initial 
experiments  indicate  that  it  will  be  possible  to  employ 
the  Eidophor  in  random  scan.  Skew  line  segments  have 
been  generated  at  various  positions  and  angles  using  a 
standard  black  and  white  Eidophor.  None  of  the  ex¬ 
pected  “sag”  in  the  oil  film  was  noticed  although  the 
smoothing  effeet  of  the  raster  scan  was  absent.  Modifi¬ 
cation  to  the  Schlieren  system  will  probably  be  neces¬ 
sary  to  make  the  system  equally  sensitive  to  horizontal 
or  vertical  lines.  If  this  study  is  successful  the  need 
for  complex  scan  conversion  and  digital-to-video  buffer¬ 
ing  equipment  will  be  eliminated. 

A  second  problem  area  of  the  commercial  Eidophor 
for  military  application  is  in  the  area  of  reliability.  For 
military  field  use  it  is  desirable  to  have  minimum  main¬ 
tenance  requirements,  repair  times  and  maximum  opera¬ 
tion  time  between  maintenance  and  failures.  Since  the 
Eidophor  contains  many  mechanically  moving  parts  its 
reliability  is  theoretically  lower  than  a  comparable  non¬ 
mechanical  projector,  e.g.,  one  that  is  fully  electronic. 
Also  the  Eidophor  cassette  contains  both  the  electron 
gun  and  the  oil  film  control  layer.  This  contributes  to 
the  need  for  continuous  vacuum  pumping  and  to  short 
lifetimes  of  the  electron  gun  cathodes.  To  obtain  mini¬ 
mum  down-time  a  turret  of  three  cathode  assemblies  is 
contained  in  each  gun  assembly.  As  a  cathode  fails  a 
new  one  can  be  rotated  in  place,  under  vacuum,  in  a 
few  minutes.  This  is,  at  best,  a  temporary  solution  to 
the  problem  from  a  military  users  standpoint.  Improve¬ 
ment  in  cathode  life  is  required  if  the  Eidophor  is  to 
be  widely  employed  in  field  use. 

Problems  of  chemical  deterioration  of  the  oil  under 
electron  beam  bombardment  is  minimized  by  renewing 
the  oil  film  from  a  filtered  supply  as  the  mirror  rotates 
at  one  revolution  every  2x/i  minutes.  Polymerization  of 
the  oil  does  occur,  however,  and  periodic  replacement 
of  the  oil  is  required. 

B.  Pin  matrix  oil  film  light  valve 

While  RADC  continues  to  explore  ways  of  improving 
available  display  projectors,  it  also  seeks  new  ap¬ 


proaches  in  display  technology.  Conceivably  these  new 
techniques  may  provide  all  the  present  capabilities  of 
light  valve  projectors  and  additionally  provide  those 
desired  characteristics  that  current  devices  lack.  The 
pin  matrix  oil  film  light  valve  is  one  such  technique. 
The  feasibility  of  this  device  to  perform  as  a  highly  re¬ 
liable  random  access  light  valve  with  expected  charac¬ 
teristics  similar  to  the  Eidophor,  has  been  demonstrated. 
The  technique  utilizes  the  principle  of  deformation  of 
a  smooth  oil  film  to  produce  an  optical  image.  However, 
the  Schlieren  optical  system  and  the  method  employed 
to  deform  the  oil  layer  differ  from  the  Eidophor. 

The  pin  matrix  light  valve  consists  of  a  permanently 
sealed  cathode  ray  tube  incorporating  a  special  wire 
mosaic  (pin  matrix)  faceplate.  Figure  19  is  a  conceptual 
drawing  illustrating  this  technique.  Detail  of  the  pin 
matrix  faceplate  is  shown  in  Figure  20.  The  control  layer 
is  illuminated  through  an  aperture  of  a  45°  mirror.  If 
the  oil  film  is  smooth  the  dielectric  mirror  beneath  the 
oil  reflects  the  incident  light  back  through  the  aperture 
to  the  light  souree.  If  a  deformation  is  introduced  in 
the  layer,  the  incident  light  is  refracted  resulting  in  a 
disturbed  ray  of  light  which  strikes  the  45°  mirror.  An 
image  of  the  deformation  then  appears  on  the  screen. 

To  produce  a  display,  deformations  are  introduced 
on  a  point  by  point  basis  and  are  accomplished  by 
means  of  electrostatic  fields.  The  fields  are  established 
between  each  of  the  isolated  pins  of  the  faceplate  and 
the  external  electrode  structure.  A  negative  potential 
is  provided  to  the  external  electrode  by  a  power  supply. 
The  pins  are  charged  positively  to  establish  an  electro¬ 
static  field  around  each  pin.  The  flood  guns  cover  the 
entire  mosaic  with  charge.  Depending  on  the  initial 
pin  voltage  and  the  energy  of  the  flood  gun  electrons, 
each  pin  charges  to  one  of  two  stable  potentials.  By 
choosing  the  proper  secondary  emitter  and  using  a  sec¬ 
ondary  collector,  the  write  gun  can  be  used  to  switch 
each  individual  pin  to  the  desired  stable  state.  Variable 
persistence  and  storage  are  possible  when  using  the 
tube  in  this  mode. 

Another  flood  gun  energy  results  in  the  pins  having 
only  one  stable  state.  The  write  gun  charges  the  pins 
to  some  unstable  potential  and  the  flood  gun  returns 
them  to  the  stable  state.  This  mode  would  be  used  for 
dynamic  displays  where  no  storage  of  the  image  is  de¬ 
sired. 

In  either  case  the  display  can  be  randomly  addressed 
since  the  oil  film  is  smoothed  by  surface  tension  only. 
There  is  no  direct  deposition  of  high  energy  electrons 
onto  the  oil  Iayer^  hence  no  chemical  deterioration  of 
the  oil  is  expected. 

Reliability  of  a  projector  utilizing  this  technique 
should  be  high.  This  is  based  on  the  fact  that  the  tube 
is  essentially  a  standard,  permanently  sealed  cathode 
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Figure  19 — Pin  matrix  light  valve  concept 


ray  tube  differing  only  in  the  faceplate  construction. 
Since  the  control  layer  is  external  to  the  vacuum,  this 
source  of  cathode  contamination  is  eliminated.  Also  the 
system  is  purely  electronic  which  leads  to  higher  reli¬ 
ability  since  no  moving  parts  are  required  for  operation. 

During  the  first  year's  study,  the  contractor*  demon¬ 
strated  feasibility  of  using  this  type  tube  to  produce 
discrete  deformations  of  the  oil  control  layer.  The  reso¬ 
lution  of  the  write  gun  used  in  the  breadboard  tube 
was  too  low  to  charge  an  individual  pin,  however,  in¬ 
dividual  pin  images  do  appear  on  the  screen  indicating 
single  pin  resolution  is  possible.  Variable  persistence 
has  also  been  experimentally  verified.  Interpin  capaci¬ 
tance  is  a  factor  determining  how  fast  the  write  beam 
can  charge  an  individual  pin.  The  data  rate  capability 


♦Air  Force  Contract  AF30( 602) -3646,  Random  Access  Light 
Valve  Study,  Stroniberg  Carlson  Corporation,  1966. 


of  the  tube  therefore  depends  upon  the  choice  of  pin 
size,  spacing  and  the  dielectric  constant  of  the  insulating 
glass  substrate.  Further  study  is  required  to  determine 
the  optimum  matrix  geometry  and  construction. 

The  optical  illumination  system  used  for  the  bread¬ 
board  model  was  adapted  from  another  projector.  For 
this  reason  contrast  and  efficiency  measurements  were 
not  as  high  as  theoretically  predicted.  Also  contributing 
to  degradation  of  optical  quality  of  the  display  were 
scratches  in  the  pin  matrix  faceplate  mirror  surface  and 
difficulty  in  establishing  a  completely  smooth  dust  free 
oil  layer. 

The  pin  matrix  faceplate,  specially  designed  and  de¬ 
veloped  for  this  technique,  was  1  inch  in  diameter  with 
individual  pins  of  2  mils  diameter  spaced  4  mils  on 
center.  The  difficulty  of  achieving  proper  pin  size  and 
spacing  while  maintaining  vacuum  integrity  between  the 
pins  and  the  glass  cladding  structure  is  no  longer  a 
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major  problem.  Present  contractual  studies**  anticipate 
use  of  2  inch  matrices. 

The  contractor  has  developed  techniques  for  polish¬ 
ing  the  mixed  glass-metal  faceplate  and  for  depositing 
of  electrodes,  secondary  emitters,  mirrors  and  protective 
coatings. 

Areas  of  concern  in  the  present  study  arc:  improve¬ 
ment  of  the  quality  of  large  mosaics,  and  improvement 
of  mirror  and  buffer  coatings.  Optimization  of  the 
electron  optics  to  obtain  single  pin  excitation  and 
further  study  of  the  secondary  emitter  requirements  arc 
also  being  pursued  (Hamann,  O.  F.  1966). 

**Air  Force  Contract  AF30(602)-4286  Pm  Matrix  Light 
Valve  Si udy,  Stromberg  Carlson  1966  (in  progress). 


C.  Solid  state  light  valve 

The  two  previously  discussed  light  valves  employ 
oil  film  control  layers  and  Schlicrcn  optics  to  produce 
large  screen  projection  displays.  The  solid  state  light 
valve  utilizes  an  electro-optic  crystal  and  polariscope 
optics  to  accomplish  this  function. 

The  original  concept  of  using  electro-optic  crystals 
in  a  light  valve  application  dates  back  to  1934  when 
M.  Von  Ardenne  invented  the  Ardenne  Tube  for  the 
German  Post  Office.  For  a  variety  of  reasons,  the 
technique  was  not  extensively  explored  for  display  ap¬ 
plication  until  recent  years.  In  1961,  RADC  took  an 
active  interest  in  the  Solid  State  Light  Valve  and  has 
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since  pursued  it  with  in-the-house  and  contractual  ef¬ 
forts  (See  footnotes  1-5). 

The  goals  of  these  studies  are  directed  toward  a 
sealed  electro-optic  projector  tube  capable  of  achieving 
a  practical  real-time  display.  A  sealed  tube  was  fabri¬ 
cated  and  demonstrated  projection  of  low  resolution 
TV  images  at  low  light  levels.  The  results  of  these  stud¬ 
ies  have  indicated  the  basic  feasibility  of  this  technique, 
however  further  improvements  arc  required  before  a 
practical  device  is  realized. 

In  concept,  the  polariseope  optics  used  in  this  light 
valve  consists  of  two  plane  polarizers  whose  preferred 
axes  are  oriented  orthogonally.  Light  incident  on  the 
pair  is  plane  polarized  on  passing  through  the  first 

1  Solid  State  Light  Valve  Study  1964. 7 

2  Air  Force  Contract  AF30(602)-2645  Solid  State  Beam 
Controlled  Light  Modulator,  Motorola  Corp,,  1963. 

3  Solid  State  Light  Valve  Study  1 965. 6 

4  Air  Force  Contract  AF30  (602)-3263,  Autonetics  Corp. 
1965. 18 

5  Air  Force  Contract  AF30(602-3730,  Autonetics  Corp.  1966, 
Final  Report  not  yet  published. 


polarizer  and  is  stopped  upon  reaching  the  second 
polarizer  (analyzer).  If  one  inserts  between  the  polar¬ 
izers  of  the  polariseope  an  optically  active  clement  then 
effectively  the  plane  of  the  polarized  light  is  rotated  and 
a  component  of  the  light  passes  the  analyzer. 

In  the  display  scheme  envisioned,  an  eleetro-optie 
crystal  is  placed  in  a  collimated  light  beam  between 
the  polarizer  and  analyzer  (Fig.  21 ).  With  no  electrical 
charge  applied  to  the  crystal,  it  is  optically  inactive 
and  the  light  is  unaffected  by  its  presence.  Therefore, 
no  light  passes  the  analyzer.  When  an  electric  field  is 
applied  across  the  faces  of  the  crystal  in  a  direction 
parallel  to  the  light  beam,  changes  occur  within  the 
crystal  making  it  optically  active.  The  plane  polarized 
light  is  changed  to  elliptical  polarization,  thus  a  com¬ 
ponent  of  light  can  pass  the  analyzer.  This  effect  in 
eleetro-optie  crystals  is  known  as  the  Pockcls1 2 3 4 5  or  Linear 
Eleetro-optie  Effect.  The  described  concept  represents 
an  optical  shutter  whereby  the  light  that  passes  the 
analyzer  depends  on  the  bias  voltage  across  the  crystal. 

By  using  an  electron  beam  to  produce  localized 


Figure  21 — Solid  state  light  valve  concept 
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fields,  the  effect  becomes  usable  as  a  display  device. 
The  beam  is  scanned  and  modulated  to  deposit  an 
image  charge  pattern  with  a  one  to  one  correspondence 
to  the  desired  optical  display.  This  results  in  a  point 
by  point  shuttering  of  the  light  in  proportion  to  the 
applied  charge  and  an  image  is  produced  at  the  screen. 
It  can  be  seen  that  the  technique  can  be  applied  in  cither 
raster  or  random  scan  modes. 

Figure  21  depicts  a  transmissive  mode  of  operation 
of  the  solid  state  light  valve.  This  mode  of  operation 
requires  a  voltage  across  the  KD.P  (Potassium  Di¬ 
deuterium  Phosphate)  crystal  of  about  3  KV  to  produce 
maximum  brightness.  In  addition,  it  requires  an  op¬ 
tically  transparent  conductive  coating  on  the  rear  face 
of  the  crystal  for  a  ground  plane  and  signal  return. 
This  mode  of  operation  is  presently  being  considered 
in  experimentation.  However,  recently  acquired  data 
indicates  sensitivity  of  the  electro-optic  effect  to  changes 
in  temperature.  This  may  present  a  technical  problem 
using  this  configuration  in  conjunction  with  higher  light 
flux  illumination.  Despite  lack  of  optimization,  a  tube 
of  this  type  demonstrated  150-  200  lines/in.  resolution 
and  100:1  contrast  ratio  utilizing  a  standard  TV  input 
signal  and  low  intensity  light  flux. 

An  alternative  mode  of  operation  is  the  reflex  con¬ 
figuration.  This  tube  requires  only  half  the  voltage 
across  the  electro-optic  crystal  to  produce  the  same 
brightness  at  the  screen  as  the  transmissive  tube.  Here 
the  transparent  conductor  can  be  replaced  with  a 
metallic  mirror  and  special  reflex  optics  introduced  to 
effect  projection  of  the  desired  image.  Additionally  the 
reflex  tube  can  be  designed  for  case  of  temperature  con¬ 
trol  of  the  crystal  if  temperature  is  discovered  to  be  a 
critical  parameter  of  the  device. 

Electro-optic  crystals  used  in  this  technique  must 
be  single  crystalline,  optically  clear,  strain  free  and 
elcctro-optically  uniform.  For  application  these  crystals 
must  be  cut  into  Z-0°  plates  and  optically  polished  to 
thicknesses  of  approximately  0.010  inches  Large  cross 
sections  (approximately  2"  X  2")  are  needed  in  order 
to  achieve  high  resolution  images.  Crystals  currently 
available  do  not  possess  all  the  aforementioned  desired 
qualities.  Non-uniformities  in  electro-optic  response, 
strains  and  small  cross  sections  arc  of  major  concern 
for  practical  application.  Of  the  available  materials  ex¬ 
amined,  KDP  (Potassium  Dihydrogcn  Phosphate)  and 
KD2P  (Potassium  Dideuterium  Phosphate)  arc  best 
suited  for  this  technique.  RADC  plans  to  study  methods 
of  growing  better  quality  KDP  and  KD,P  crystals  of 
large  cross  section.  Also  of  interest  arc  new  materials 
which  show  an  improvement  over  KDP  and  KD,P 
particularly  in  the  voltage  required  to  produce  maxi¬ 
mum  light  intensity.  (Half-wave  retardation  voltage.) 
If  this  voltage  is  decreased,  the  electron-optics  and 


resolution  problems  become  less  difficult.  One  possible 
crystal  improvement  is  the  deuteration  of  KDA  (Po¬ 
tassium  Dihydrogen  Arsenate).  If  this  follows  the  pat¬ 
tern  set  by  other  deuterated  crystals,  its  half-wave 
retardation  will  be  less  than  that  of  KD2P. 

Before  a  practical  solid  state  light  valve  evolves  fur¬ 
ther  studies  are  required.  These  studies  include  tem¬ 
perature  effects,  the  seriousness  of  the  crystal  out- 
gassing  problem,  and  the  display  parameters  achievable 
in  the  optimized  tube  design.  Development  of  new  pro¬ 
cessing  techniques  are  needed  to  polish  crystals  to 
higher  optical  tolerances  in  thicknesses  less  than  0.010". 
Further  investigation  of  sealant-protective  coatings  is 
required.  Also  a  means  must  be  found  to  reliably  de¬ 
posit  the  conductive  electrode  on  the  crystal,  since  ex¬ 
periments  have  indicated  poor  adhesion  of  the  elec¬ 
trodes  and  changes  in  their  conductivity  after  a  period 
of  tube  operation. 

CONCLUSION 

The  primary  purpose  of  this  paper  has  been  to 
describe  several  mechanisms  which  have  application 
to  the  generation  of  large  screen  displays.  It  is  obvious 
that  various  ancillary  equipments  arc  required,  in  addi¬ 
tion  to  the  display  generation  device,  to  complete  the 
makeup  of  a  display  system.  The  reason  for  making 
this  obvious  point  is  to  emphasize  that  in  the  final 
analysis  of  the  display  system  configuration,  the  most 
important  consideration  may  very  well  be  governed  by 
availability,  cost  and  performance  of  this  ancillary 
equipment.  For  example,  some  of  the  work  discussed 
has  centered  on  a  sequential  (raster  scan)  mode  of 
operation.  Such  a  system  will  require  digital  to  video 
conversion  to  mate  the  display  device  to  the  computer 
input.  A  more  practical  approach  might  be  to  consider 
selective  address  (random)  mechanisms  in  the  display 
device,  which  mate  directly  with  the  computer  input. 
Research  is  presently  underway  in  this  area. 

Finally,  the  schemes  described  and  models  fabri¬ 
cated  are  presently  categorized  as  being  research  devices 
(with  the  exception  of  the  Eidophor).  A  considerable 
amount  of  refinement  will  be  required  in  order  that 
existing  reliability  and  maintainability  specifications  can 
be  met  for  field  applications. 
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Military  command  information  systems  from  the  user’s  point  of  view 


by  Joseph  G.  Carley,  Jr.,  Lt.  Colonel,  USAF 

Headquarters  United  States  Air  Force 
Washington,  D.C. 

INTRODUCTION 

When  an  automated  command  and  control  system 
reaches  operational  status,  the  term  “USER”  assumes 
a  more  specific  meaning  than  the  interpretation  nor¬ 
mally  applied  by  the  technical  and  development  com¬ 
munity.  According  to  the  latter  interpretation,  any  com¬ 
mand  or  agency  which  acquires  and  uses  a  system  is 
referred  to  as  a  “USER.” 

Generally  speaking,  the  Department  of  Defense  con¬ 
cept  for  a  command  and  eontrol  system  encompasses 
the  various  elements  that  are  required  for  the  com¬ 
mander  to  eontrol  his  forces  and  resources.  This  in¬ 
cludes  all  of  the  associated  facilities,  people  and  equip¬ 
ment  which,  when  tied  together  with  procedures,  form 
a  total  integrated  system.  Of  equal  importance  are  the 
functions  necessary  to  make  the  system  operate  ef¬ 
fectively.  In  a  headquarters  environment,  broad  par¬ 
ticipation  by  the  functional  staff  is  obviously  essential. 

The  automated  command  and  eontrol  system,  of 
course,  is  but  one  element  of  the  total  command  and 
control  system  of  any  agency,  command  or  service. 
However,  it  is  rapidly  becoming  a  key  element  and 
therefore  care  must  be  taken  to  insure  it  is  appro¬ 
priately  responsive  to  the  needs  of  the  staff  and  com¬ 
mander.  These  needs  must  be  translated  into  facilities, 
equipment  and  software  design.  It  is  appropriate,  then, 
for  the  people  who  arc  directly  responsible  for  operating 
the  system  to  look  upon  the  staff  and  commander  as 
the  real  “USERS”  of  the  system.  This  introduces  a 
special  category  of  personnel  and  organizations  which 
must  have  a  strong  voice  in  command  and  eontrol 
matters:  The  SYSTEM  OPERATORS. 

Uniformally  throughout  the  Air  Force  the  staff 
operations  officer  is  assigned  the  responsibility  for 
managing  and  operating  the  command  center  complex. 
Since  the  computer  system  is  an  integral  part  of  this 
complex,  the  operations  officer  is  placed  in  the  role  of 
SYSTEM  OPERATOR. 

Although  the  use  of  the  term  “USER”  in  the  title 
of  this  paper  is  generally  correct,  it  is  specifically  from 


the  perspective  of  the  System  Operator  that  the  ideas 
and  opinions  are  expressed  in  the  following  paragraphs. 

The  headquarters  level  system 

Command  eontrol  and  communications  systems  ean 
be  divided  into  four  major  categories:  Communications; 
surveillance;  direct  eontrol  (taetieal  and/or  strategic 
eontrol  of  airborne  aircraft),  and  semi-automated  fixed 
facility,  headquarters  level  command  center  systems. 
The  first  three  categories  of  systems  are  looked  upon  as 
special  purpose  in  nature  and  usually  function  remotely 
from  the  staff  officer.  It  is  the  final  category  whieh 
currently  demands  our  attention,  consumes  much  of  our 
staff  energy  and  convokes  diseussion  from  all  quarters. 
We  have,  therefore,  confined  our  diseussion  to  this  par¬ 
ticular  area. 

The  headquarters  automated  command  and  eontrol 
system  has  been  cloaked  with  a  “romance  of  words” 
by  technical  writers,  policymakers,  system  designers, 
system  developers  and  a  myriad  of  self-styled  command 
and  eontrol  experts.  It  is  said  that  systems  must  be 
developed  from  an  “incremental,  evolutionary  ap¬ 
proach”;  that  they  must  be  “compatible”  with  one  an¬ 
other;  that  standardization  must  be  achieved  wherever 
possible,  and  that  they  must  be  “immediately  and  di- 
reetly  responsive”  to  the  commander  for  whom  the 
system  is  designed. 

These  words  and  phrases  are  found  in  policy  letters, 
concepts  of  operations^  and  professional  papers.  The 
policy  letters  are  intended  to  provide  guidance  to  the 
commander  contemplating  the  acquisition  of  an  auto¬ 
mated  system  and  to  the  technical  agency  or  contractor 
having  design  and  developmental  responsibilities.  How¬ 
ever,  these  particular  words  and  phrases  are  largely 
undefined  in  the  specific  sense.  The  reader  or  listener 
applies  his  own  interpretation  as  to  their  exact  meaning 
depending  on  his  background,  parochial  interests,  or¬ 
ganizational  loyalties  or  position.  The  result  is  con¬ 
structive  controversy  in  the  approach  to  command  and 
eontrol  policy,  management,  development,  acquisition, 
operation  and  utilization. 
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During  recent  years  the  din  of  command  and  control 
discussion  has  been  raised  a  few  decibels  by  a  growing 
chorus  of  Air  Force  command  and  control  system 
operators.  The  words  of  the  policymakers  have  begun 
to  crystalize  into  specific  meanings  to  some  people  in 
this  system  operator  group.  As  the  definitions  evolve, 
so  also  evolve  concepts:  Concepts  for  approaches  to 
design;  concepts  for  development;  but  primarily  con¬ 
cepts  for  operation.  The  conceptual  points  of  view  of 
at  least  one  operator  are  presented  herein.  The  subject 
matter  is  examined  first  from  the  specific  aspects  of 
the  development,  acquisition,  operation  and  use  of  a 
single  system  and  then  broadened  to  provide  a  per¬ 
spective  and  an  overview  of  Air  Force  automated  (head¬ 
quarters  level)  command  and  control  systems  as  a 
TOTAL  RESOURCE.  The  approach  to  discussion  is 
made  by  way  of  providing  a  system  operator’s  defini¬ 
tion  of  some  of  the  words  and  phrases  previously 
referenced. 

The  incremental ,  evolutionary  approach 

There  is  complete  agreement  between  policymakers, 
system  operators,  designers,  developers  and  systems 
engineers  that  an  automated  command  and  control  sys¬ 
tem,  or  any  computer  system,  must  be  well  documented 
in  specification  form  before  actual  development  begins. 
The  most  recent  policy  statement  concerning  design 
specification  was  published  on  29  July  1966  by  the 
Secretary  of  Defense  in  his  defense  memorandum, 
Management  and  Use  of  the  Electronic  Computer , 
Washington,  D.C.:  Department  of  Defense,  pg.  3,  who 
said: 

“We  must  select  and  acquire  new  or  replace¬ 
ment  computers  only  after  systems  have  been  re¬ 
designed  to  make  full  use  of  the  improved  capabil¬ 
ities  of  later  model  hardware,  and  then  only  where 
there  are  proven  cost  benefits.  In  these  cases , 
systems  redesign  and  programming  will  be  accom¬ 
plished  prior  to  delivery  of  any  equipment.  Fur¬ 
ther,  computers  will  not  be  selected  until  the 
performance  of  the  complete  hardware/ software 
package  required  in  the  systems  specification  and 
request  for  proposal  has  been  clearly  demonstrated 
by  either  a  full-scale  or  benchmark  test  .” 

Depending  on  the  interpretation  of  the  reader,  the 
policy  and  sound  engineering  principle  for  fully  docu¬ 
mented  systems  could  be  in  conflict  with  the  policy  of 
incremental,  evolutionary  development.  Let  us  explore 
the  possible  meanings  of  the  latter  policy. 

One  interpretation  could  require  the  development  of  a 
series  of  systems,  with  each  system  in  the  series  designed 
as  a  complete  entity  and  with  each  system  representing 
a  controlled  step  upwards  in  power,  capacity  and  com¬ 
plexity.  Theoretically,  the  command  center  would  even¬ 
tually  reach  the  panacea  in  automated  capability. 


A  major  advantage  to  this  approach  would  be  the 
learning  curve  to  which  the  operators,  staff  users  and 
technical  support  agencies  would  be  exposed.  Each  sys¬ 
tem  in  the  series  would  benefit  by  the  design  errors  or 
deficiencies  of  its  predecessors.  However,  such  an  ap¬ 
proach  would  place  the  operator  in  the  position  of 
operating,  maintaining  and  improving  one  system  while 
designing  and  developing  its  replacement.  His  resources 
would  be  divided.  Further,  the  period  of  overall  develop¬ 
ment  activity  would  be  long,  tedious  and  expensive. 

A  seeond  interpretation  is  based  on  the  idea  of 
definition  and  specification  from  the  “total  system” 
approach.  This  approach  requires  the  determination  (to 
the  best  judgment  of  the  system  designers)  of  maxi¬ 
mum  future  requirements  for  equipment  in  such  areas 
as  computing  power,  display,  core  and  mass  storage, 
speed  and  capacity.  In  the  software  area  all  elements, 
including  both  support  and  functional  applications, 
must  be  defined  and  documented,  designed  and  de¬ 
veloped.  Once  completed,  the  system  is  turned  over  to 
the  system  operator  at  which  point  the  incremental 
growth  process  begins. 

This  is  considered  the  classical  approach  and  the 
one  most  likely  to  be  recommended  by  the  system  en¬ 
gineer.  But  from  the  operator’s  point  of  view,  there  are 
some  important  problems  associated  with  the  total  sys¬ 
tem  approach  which  must  be  examined. 

In  the  first  place,  if  a  command  and  control  system 
is  to  be  developed  in  consonance  with  the  Department 
of  Defense  command  and  control  concept,  the  system 
must,  by  necessity  ^  result  in  a  complexity  of  interacting 
files  and  programs.  These  files  and  programs  support 
multi-users  working  on  problems  and  solutions  which 
are  influenced  and  even  founded  on  information,  deci¬ 
sions,  factors  and  operational  requirements  derived 
from  the  various  functional  activities. 

As  a  practical  example,  the  support  functions  such 
as  transportation,  logistics,  personnel  and  medical  can¬ 
not  begin  the  planning  process  for  any  contingency 
operation  until  the  force  requirements  are  specified.  In 
turn,  the  force  requirements  cannot  be  determined  until 
the  threat  is  analyzed  and  both  defensive  and  offensive 
weaponry  are  selected,  the  length  of  engagement  fore¬ 
cast  and  probable  attrition  and  expenditure  rates  are 
computed. 

Once  an  operation  gets  under  way  the  means  must  be 
provided  which  will  permit  the  appropriate  staffs  at 
all  levels  to  monitor  current  operations,  to  determine 
the  validity  of  planning  factors  and  to  analyze  and 
establish  future  requirements. 

Obviously  the  basic  automated  system  requirement 
calls  for  a  planning  oriented  capability  designed  around 
an  ability  to  develop  accurate  operational  profiles. 
These  profiles  directly  influence  one  another  and  at 
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several  levels  of  command  must  be  combined  into  an 
overall  profile  of  the  total  operational  requirement. 

This  general  description  of  the  problem  is  “old  hat” 
to  the  military  staff  officer.  Throughout  military  history 
these  planning  functions  have  been  performed  with 
varying  degrees  of  success  and  have  been  a  strong 
influence  in  the  results  of  war.  Nor  is  it  to  be  considered 
a  profound  announcement  that  “automation  of  opera¬ 
tional  profiles  is  within  the  technical  state-of-the-art 
today.”  However,  when  the  design  goals  of  an  auto¬ 
mated  system  encompass  this  total  operational  spectrum, 
the  designers  are  faced  with  an  overall  problem  which 
may  yet  be  too  complex  for  assurance  of  a  satisfactory 
product. 

One  of  the  major  problems  of  system  design  is  the 
translation  of  the  ideas  and  experience  of  functional 
staff  specialists  into  an  understandable  form  for  the 
systems  designer.  As  the  systems  design  grows  in  com¬ 
plexity  and  size,  there  is  a  comparable  increase  in  the 
numbers  of  people  required  to  cover  the  tasks  involved. 
The  determination  of  these  individual  functional  re¬ 
quirements  and  the  accomplishment  of  associated  de¬ 
sign  activities  must  be  performed  under  an  overall  and 
integrated  management  scheme.  The  resultant  manage¬ 
ment  task  is  best  described  as  severe.  Top  man¬ 
agement  personnel  find  it  extremely  difficult  to  cor¬ 
relate  all  of  the  activities  in  such  a  manner  that  the 
original  concept  of  the  particular  system  design  remains 
inviolate. 

The  period  of  time  required  to  develop  studies, 
prepare  detailed  specifications  and  begin  actual  develop¬ 
ment  normally  extends  beyond  the  average  tours  of 
duty  of  the  functional  staff  officers  and  even  beyond 
the  tenure  of  the  management,  analysis  and  program¬ 
ming  staffs  of  the  contractors  and  other  technical  sup¬ 
porting  agencies.  With  the  rotation  of  personnel  new 
ideas  and  influences  are  introduced  in  the  direction  of 
systems  development.  Hence,  continuity  in  develop¬ 
mental  management  and  direction  is  periodically  jeop¬ 
ardized.  Further,  the  original  concept  is  in  constant 
danger  of  gradual  erosion  from  many  other  causes, 
including:  Budget  cuts,  technical  problems,  policy 
changes  and,  last  but  most  important,  changes  in  mili¬ 
tary  missions  and  requirements. 

The  development  of  the  Headquarters  USAF  Com¬ 
mand  and  Control  System  was  approached  in  a  com¬ 
bination  of  alternatives  one  and  two.  An  initial  small 
scale  “off-the-shelf”  computer  was  placed  in  the  com¬ 
mand  post  and  operated  with  a  software  system  which 
served  as  a  small  model  of  the  “total  system”  design. 
Even  though  the  system  was  limited  in  capability,  it 
was  considered  a  success  and  was  accepted  as  the  basic 
design  for  the  next  step.  The  small  scale  system 
shouldered  the  staff  support  task  and  was  treated  as  an 


operational  system,  during  which  time  valuable  train¬ 
ing  and  operational  experience  was  gained.  The  same 
system  was  also  literally  exported  to  five  Air  Force 
major  commands  with  surprising  success.  (More  about 
this  later.) 

Development  of  the  second  system  was  undertaken 
on  the  operational  site  in  the  USAF  Command  Post 
at  Headquarters  USAF  and  was  subjected  to  all 
of  the  erosion  processes  referred  to  previously.  The 
most  significant  influence  came  in  the  form  of  US 
military  action  in  Southeast  Asia. 

The  Headquarters  USAF  “total”  system  had  been 
under  development  for  several  years  when  hostilities 
began  and  final  realization  of  the  design  capability  was 
still  several  years  in  the  future.  The  smaller  system 
was  pressed  into  the  role  of  supporting  staff  require¬ 
ments  brought  on  by  Southeast  Asia.  “Inhouse”  tech¬ 
nical  personnel  who  could  be  spared  from  the  larger 
system  development  task  were  concentrated  on  improv¬ 
ing  the  operational  system.  Unfortunately,  only  a  small 
portion  of  the  total  “inhouse”  force  could  be  spared 
from  the  developmental  activities  without  impairing 
production  schedules. 

It  is  important  to  note  that  neither  the  operational 
nor  developmental  system  designs  could  accommodate 
the  new  requirements  generated  by  Southeast  Asia.  It 
would  have  been  difficult  to  forecast  that  modern  war 
management  would  require  the  examination  of  such 
large  volumes  of  raw  operational  data,  or  that  the 
evaluation  spectrum  would  venture  into  an  infinity  of 
requirements,  or  that  they  would  be  demanded  with 
such  rapidity  from  the  highest  levels  of  national  author¬ 
ity.  This  does  not  mean,  however,  that  the  system  de¬ 
signers  could  not  have  accommodated  these  require¬ 
ments  or  that  the  functional  specialists  involved  were 
shortsighted  and  failed  to  recognize  the  growth  potential 
of  an  automated  system.  As  a  matter  of  fact,  the  re¬ 
quirements  for  detailed  information  at  such  high  levels 
arc  largely  disassociated  from  the  general  theorv  that 
“automation  in  itself  creates  a  demand  for  detail.”  The 
unanticipated  actual  requirements,  in  the  opinion  of 
the  author,  have  grown  out  of  necessity  because  of  the 
world  political  situation  and  the  requirements  for  “tight” 
management  and  responsiveness  in  supporting  US 
Forces  in  Southeast  Asia. 

Establishing  the  reporting  system  for  producing 
the  required  detailed  raw  data  and  the  basis  from 
which  the  necessary  computer  programs  could  be  de¬ 
veloped  necessitated  lengthy  and  continuous  collective 
efforts  on  the  part  of  staff  personnel  from  the  Joint 
Staff,  the  Services,  the  Commander-in-Chief  Pacific  and 
his  service  components. 

It  was  felt  in  the  beginning  that  the  task  of  co¬ 
ordination  might  prevent  early  achievement  of  a  re- 
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porting  system  which  could  satisfy  the  requirements  and 
interests  of  the  various  levels  of  command  and  staff 
and  of  the  National  Command  Authorities.  Consider¬ 
ing  the  magnitude  of  the  task,  however,  it  was  com¬ 
pleted  in  record  time.  The  Headquarters  USAF  opera¬ 
tional  system  analysis  and  programming  contingent  em¬ 
barked  on  a  maximum  effort,  which  continues  today, 
to  provide  a  capability  for  appropriate  Air  Force  per¬ 
sonnel  to  review  and  analyze  the  detailed  air  operations 
reports  flowing  in  from  Southeast  Asia. 

Although  the  accomplishment  of  the  “inhouse”  per¬ 
sonnel  thus  far  can  be  looked  upon  with  a  measure  of 
pride,  the  actual  requirement  has  only  been  partially 
satisfied.  There  is  continuous  pressure  on  the  system 
operator  to  produce  greater  numbers  of  capabilities 
within  a  much  shorter  time  frame  than  is  possible  for 
the  numbers  of  people  available. 

The  above  opinions  and  the  Headquarters  USAF 
example  leads  us  to  some  fairly  firm  conclusions  on  the 
‘‘total  system”  approach  to  design  and  development. 
For  example,  there  can  be  no  argument  that  command 
and  control  systems  must  be  based  on  an  integrated- 
function  concept  to  insure  correlated  staff  participation 
in  the  command  and  control  function  and  to  make  the 
most  effective  use  of  the  power  of  a  computer. 

On  the  other  hand,  to  arrive  at  an  integrated  system 
in  “one  pass  at  the  target”  may  be  attempting  to  do  too 
mueh  at  one  time.  The  calendar  time  required  for  sys¬ 
tem  development  is  excessive  which  exposes  the  con¬ 
cept  to  erosion  and  vacillation  processes  and  the  func¬ 
tional  design  itself  may  be  overtaken  by  events.  Lengthy 
on-site  development  activities  disrupt  command  center 
operations  and  strain  the  patience  of  command  center 
personnel,  operational  supervisors  and  staff  users  as 
well. 

The  system  operator  must  get  an  operational  system 
as  soon  as  possible  so  that  he  can  respond  to  his  im¬ 
mediate  requirements.  His  system  must  incorporate  all 
of  the  available  and  worthwhile  “state-of-the-art”  tools 
which  will  provide  for  flexible  and  rapid  response.  The 
system  operator’s  attitude  can  be  reduced  to  a  simple 
statement:  “Get  your  head  out  of  the  clouds  and  show 
me  some  results — now!” 

If  we  rule  out  the  series  of  systems  and  total  system 
approaches,  you  may  ask:  “Where  do  we  go  from 
here?”  This  is  a  valid  question  and  it  must  be  answered. 

There  should  be  five  objectives  in  the  design  and 
development  of  an  automated  command  and  control 
system: 

•  Build  to  aeeommodate  the  Department  of  De¬ 
fense  concept. 

•  Do  not  accomplish  basic  system  design  and  de¬ 
velopment  on  an  operational  site. 

•  Get  something  operational  as  soon  as  possible. 


•  Provide  proven  and  powerful  hardware  and  soft¬ 
ware  tools  to  assist  in  rapid  modification  of  the 
functional  software  subsystem. 

•  Plan  from  an  overall  approach  to  avoid  dupli¬ 
cative  software  developmental  effort. 

The  above  objectives  can  be  accomplished  in  a 
straightforward  approach.  First,  we  must  undertake 
to  define  the  logical  design  parameters  and  hardware 
concept  for  a  system  which  will  someday  accommodate 
the  requirements  of  the  participating  functional  staff 
agencies.  Secondly,  we  must  insure  that  the  equipment 
to  be  selected  is  truly  “off-the-shelf”  with  a  proven 
reliability  history  and  will  accommodate  the  system 
design.  Once  equipment  selection  action  has  been  taken, 
however,  we  must  insure  the  overall  system  design  re¬ 
mains  within  the  design  capability  of  the  selected  equip¬ 
ment.  (This  will  avoid  questionable  “cloud  nine”  devel¬ 
opment.)  Next,  full  budgetary  support  must  be  given 
to  the  development  of  the  basic  control,  support  and 
utility  software  to  insure  that  the  required  degree  of 
flexibility  and  responsiveness  is  an  inherent  attribute  of 
the  system.  The  resulting  basic  system  should  then  be 
intailed  in  a  central  Air  Force  “overhead”  facility  and 
subjected  to  a  rigid  “shakedown”  period.  This  facility 
must  be  assigned  to,  and  under  the  direct  control  of, 
Headquarters  USAF. 

It  is  at  the  “overhead”  location  also  that  functional 
software  applications  should  be  added  to  the  basic  sys¬ 
tem,  tested  and  determined  to  be  fully  acceptable  by 
the  USING  command.  These  applications  may  be  either 
emulated  or  simulated  versions  of  the  current  opera¬ 
tional  system  at  the  command  of  primary  interest  or  new 
programs  developed  by  that  command  with  “inhouse” 
personnel  and/or  with  contract  support. 

When  a  specified  operational  base-line  is  reached, 
the  system  should  be  installed  at  the  operational  site 
and  turned  over  to  the  operator  for  application 
and  concentrated  effort  toward  rapid  growth.  This 
approach  eliminates  many  of  the  cited  deficiencies  of 
both  the  scries  of  systems  and  total  system  approaches. 
It  concentrates  initial  developmental  activity  on  a  solid¬ 
ly  based  system  whieh  can  truly  grow  in  an  evolutionary 
manner  and  in  “quantum  jumps.” 

Of  course,  this  approach  also  has  its  unique  problems 
among  which  is  the  necessity  to  define  the  parameters 
of  the  integrated  software  system  concept.  Obviously, 
there  will  be  trade-off  problems  to  be  overcome  between 
the  tendency  to  go  overboard  with  generalization  and 
modularity  in  the  basic  support  software  package  versus 
the  need  for  a  specialized  basic  software  system  to 
support  the  unique  requirements  of  command  and 
control. 

The  most  promising  aspect  of  the  proposal,  how¬ 
ever,  is  the  opportunity  to  establish  a  standard  approach 
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Air  Force-wide  for  the  basic  tools  to  be  incorporated 
into  a  system.  This  would  permit  the  concentration  of 
developmental  funds  toward  the  design  of  a  superior 
basic  system  which  could  be  amortized  through  its  use 
universally  by  the  commands  and  agencies  receiving 
acquisition  approval. 

The  elimination  of  problems  previously  discussed 
plus  an  opportunity  to  introduce  reasonable  standard¬ 
ization  in  the  development  of  Air  Force  command  and 
control  systems  causes  this  system  operator  to  support 
the  “see  it  before  you  buy  it”  approach  just  explored. 
This  leads  us  to  the  next  area  of  discussion  and  the 
definition  of  associated  popular  phrases  from  the  system 
operator's  point  of  \ie\v. 

Compatibility  and  standardization 

The  word  compatibility  is  normally  used  in  reference 
to  operational  relationships  between  systems.  In  the 
discussion  to  follow  it  will  be  shown  that  there  are  two 
valid  levels  of  compatibility:  One  directly  concerning 
operational  relationships  and  the  other  related  to  the 
economics  of  Air  Force  management  of  operational 
systems.  This  latter  interpretation  is,  in  turn,  closely 
related  to  standardization  goals  which  shall  also  be 
specified. 

Compatibility,  as  generally  applied  in  policy  papers, 
refers  to  the  ability  to  transfer  data  from  one  system  to 
another  “without  buffers/'  This  area  of  consideration 
constantly  presents  a  problem  of  communication  be¬ 
tween  the  policymakers  and  the  more  technically  orient¬ 
ed  operators.  Because  of  this  deficiency  in  technical  un¬ 
derstanding  at  some  policy  levels,  there  is  the  even 
present  danger  of  the  publication  of  unnecessary  restric¬ 
tive  policies  or  those  which  contain  wording  subject  to 
conflicting  interpretations.  It  is  extremely  difficult  to  get 
the  point  over  that  flexible  file  maintenance  (data  con¬ 
trol)  and  retrieval  systems  can  be  designed  to  respond 
rapidly  to  any  operational  reporting  requirements  levied 
by  higher  authority,  regardless  of  whether  the  directive 
originates  from  an  operational  command  authority  or  a 
higher  service  headquarters.  It  is  also  difficult  to  con¬ 
vince  these  same  personnel  that  the  overall  system 
design,  programming  language,  specific  retrieval  lan¬ 
guage  attributes  or  model  and  design  of  computer  equip¬ 
ment  have  little  to  do  with  the  ability  to  transfer  data. 
Of  course,  it  is  mandatory  that  the  data  storage  and 
retrieval  software  elements  of  the  systems  be  designed  to 
accommodate  and  be  operated  according  to  Department 
of  Defense  standards  for  Data  Elements  and  Related 
Features  (Codes)  and  Terms  of  Reference. 

However,  if  the  objective  is  to  transfer  all  operational 
data  machineto-machinc  in  digital  form,  then  a  com- 
mon-uscr  digital  communications  system  is  required. 
The  communications  system,  having  a  standard  lan¬ 


guage,  must  serve  as  a  translator  between  the  originating 
and  target  machines.  In  this  case,  translation  occurs 
even  if  the  interacting  machines  are  of  identical  make 
and  design.  Of  course,  the  common-user  communica¬ 
tions  system  requires  the  use  of  hardware  buffers  be¬ 
tween  communications  terminals  and  the  computers 
being  served. 

Although  the  elimination  of  buffers  is  a  policy  goal, 
its  achievement  may  be  easier  said  than  done.  In  the 
first  place,  command  and  control  systems  rely  heavily 
on  data  support  from  comptroller-tvpe  management- 
oriented  systems.  Departing  from  the  common-user 
communications  system  concept  (and  the  unwanted 
buffers)  would  require  standardization  of  all  interacting 
computer  systems  in  use  in  the  Department  of  Defense 
and  probably  certain  other  Federal  agencies  and  de¬ 
partments.  Further,  a  dedicated,  completely  compatible, 
communications  network  would  have  to  be  established 
between  the  systems  involved.  Obviously,  this  would 
be  unrealistic  under  current  policy  and  will  remain  so 
unless  it  is  proven  that  the  concept  of  common-user 
communications  systems  is  inadequate  for  command 
and  control  purposes. 

It  should  be  apparent  that  the  buffer  problem  is 
primarily  associated  with  hardware  and  basically  un¬ 
related  to  the  software  design  insofar  as  compatibility 
is  concerned.  Therefore,  it  can  be  concluded  that  the 
Air  Force  can  fully  satisfy  compatibility  requirements 
for  data  transfer  by  building  flexibility  into  its  systems 
in  the  manner  described. 

At  the  next  level  of  compatibility,  the  cold  facts  of 
economic  necessity  come  into  consideration.  This  orien¬ 
tation  of  the  meaning  of  compatibility  falls  within  the 
purview’  of  the  service  secretaries  who  are  charged  with 
the  responsibility  for  effective  management  of  their 
respective  departmental  resources. 

The  meaning  of  compatibility  from  the  service  as¬ 
pect,  according  to  the  author,  should  be  defined  as 
follows: 

" Standardization  of  both  hardware  and  software 
to  the  degree  necessary  to  enable  the  transfer  of 
personnel  and  computer  programs ,  as  well  as 
operational  data ,  between  operational  systems  for 
immediate  utilization ,  while  retaining  maximum 
flexibility  for  individual  commands  to  add ,  delete 
or  modify  functional  software  applications  to  satisfy 
unique  requirements  ” 

This  definition  will  support  the  overall  command  and 
control  system  concept,  enhance  the  ability  of  the  Air 
Force  to  economically  manage  its  operational  systems 
and  will  insure  the  ability  of  each  command  to  respond 
to  its  respective  operational  command  authority  and 
resource  management  channels  as  well. 

An  actual  Air  Force  example  of  the  basic  concept 
on  which  the  above  compatibility  definition  is  founded 


246  Information  System  Science  and  Technology 


is  the  AIR  FORCE  INTEGRATED  COMMAND  AND 
CONTROL  SYSTEM.  As  previously  mentioned,  by 
1965  Headquarters  USAF  had  exported  the  small-scale 
system,  developed  in  the  Headquarters  USAF  Com¬ 
mand  Post,  to  five  of  its  major  commands  of  which 
four  are  components  of  unified  commands.  This  action 
was  an  initial  step  to  provide  five  command  centers 
with  a  relatively  inexpensive  automated  capability  from 
which  expansion  could  emanate  and  through  which 
the  commands  concerned  could  be  introduced  to  the 
automated  command  and  control  concept. 

Under  the  rules  established  in  the  basic  approval 
directive,  the  EXECUTIVE  SOFTWARE  SUBSYS¬ 
TEM  (Control,  data  control,  utility  and  retrieval  com¬ 
plex)  cannot  be  modified  except  on  approval  of  Head¬ 
quarters  USAF.  The  operational  or  funetional  applica¬ 
tions  included  as  a  part  of  the  program  package  have 
been  released  for  modification  at  the  will  of  the  com¬ 
mand  concerned.  However,  the  commands  are  required 
to  coordinate  with  Headquarters  USAF  on  intended 
modifications  to  the  operational  programs  and  on  the 
development  of  new  programs.  The  single  purpose  of 
this  requirement  is  to  prevent  duplicative  efforts  between 
the  six  systems  (including  the  Headquarters  USAF 
system ) . 

Maintenance  of  the  EXECUTIVE  SOFTWARE 
SUBSYSTEM  is  retained  as  a  responsibility  of  the 
Headquarters  USAF  Command  Post  computer  program¬ 
mers.  Responsibility  for  the  maintenance  of  modified 
functional  programs  or  of  any  new  programs  is  assigned 
to  the  originating  command.  This  responsibility  in¬ 
cludes  analysis,  programming,  documentation  and 
maintenance. 

System  operator  conferences  are  held  on  a  regular 
basis  to  discuss  operational  problems  and  to  exchange 
ideas.  These  conferences  have  been  the  point  of  origin 
for  the  present  managerial  approach  to  system  config¬ 
uration  control.  There  is  wide  support  for  the  concept 
of  central  control  over  the  Executive  Program  subsys¬ 
tem  for  it  is  through  this  managerial  technique  that  the 
ability  to  transfer  operational  programs  between  the  sys¬ 
tems  is  assured.  Further,  the  commands  are  able  to 
devote  their  resources  to  the  development  of  opera¬ 
tional  capabilities  unique  to  the  command  rather  than 
having  to  expend  their  limited  technical  capability  on 
executive  subsystem  maintenance. 

The  system  operator  conferences  have  also  resulted  in 
current  emphasis  on  a  detailed  standard  for  computer 
program  documentation.  Since  various  commands  are 
using  programs  developed  by  other  commands  in  the 
system,  the  need  for  qualitative  control  over  the  stand¬ 
ard  of  documentation  is  an  absolute  necessity. 

Each  command  has  the  responsibility  for  coordinating 
with  its  respective  unified  command  headquarters  to 


insure  operational  capabilities  being  considered  by  that 
command  for  development  will,  in  fact,  be  responsive 
to  operational  requirements  of  the  unified  commander. 
Of  course,  coordination  does  not  occur  if  the  program 
is  designed  only  to  support  internal  requirements.  In 
the  event  the  unified  command  has  a  requirement  for 
operational  data  formats  which  impact  the  executive 
program,  the  Air  Force  component  commander  must 
submit  a  requirement  for  its  revision  to  Headquarters 
USAF.  It  is  interesting  to  note  that  this  form  of  basic 
modification  has  not,  as  yet,  been  necessary. 

The  Air  Force  Integrated  Command  and  Control  Sys¬ 
tem  is  planned  for  replacement  of  its  present  hardware 
in  the  1970  time  frame.  This  system  has  introduced 
a  concept  embodying  a  practical  and  economical  ap¬ 
proach  to  system  development  and  management.  It 
treats  Air  Force  Command  and  Control  Systems  as  a 
part  of  a  total  resource  for  which  the  Secretary  of  the 
Air  Force  has  the  responsibility  to  manage  and  for 
insuring  that  operational  requirements  placed  on  each 
element  of  the  total  resource  are  fully  satisfied  in 
the  shortest  possible  time,  regardless  of  whether  the 
source  of  the  requirements  are  internally  or  externally 
generated. 

It  is  appropriate  at  this  point  to  divert  for  a  few 
paragraphs  to  clarify  previous  statements  concerning 
the  requirement  for  an  “overhead”  facility  assigned 
“under  the  direct  control  of  Headquarters  USAF.” 

The  very  notion  of  standardization  carries  overtones 
of  serious  import  on  the  prerogatives  of  the  commander. 
These  prerogatives  must  be  protected  at  every  level  of 
command  to  the  maximum  extent  possible.  With  this  as 
a  controlling  factor,  justification  for  standardization  of 
automated  command  and  control  systems  is  invalid 
unless  founded  on  operational  necessity  or  when  there 
arc  overriding  economic  considerations.  Significantly, 
there  are  two  important  objectives  in  Air  Force 
command  and  eontrol:  (1)  There  must  be  continuous 
effort  toward  the  achievement  of  case  in  operational 
interaction  between  command  and  eontrol  systems;  and 
(2)  all  possible  means  must  be  fully  exploited  to 
achieve  economy  in  the  development  and  operation  of 
automated  command  and  eontrol  systems,  particularly 
in  the  area  of  software  development.  Standardization 
facilitates  both  of  these  objectives. 

The  Air  Force  Integrated  Command  and  Control  Sys¬ 
tem  has  proved  that  it  is  feasible  to  exercise  central 
management  over  standardized  elements  of  systems 
without  interfering  with  command  prerogatives  to  an 
excessive  degree.  It  has  also  validated  the  concept  that 
software  economy  can  be  achieved  through  inter-com¬ 
mand  coordination.  It  is  important  to  note,  however, 
that  standardization  has  been  imposed  only  to  the  de¬ 
gree  necessary  to  permit  mutual  support,  leaving  con- 
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sidcrable  flexibility  for  the  local  commander  to  satisfy 
unique  requirements.  It  is  also  important  to  emphasize 
that  AFICCS  management-supporting  procedures  are 
designed  to  eneourage,  rather  than  stifle,  individual  sys¬ 
tem  operator  initiative. 

Since  standardization  enables  an  environment  of 
mutual  support,  effective  mutual  support  operations  are 
therefore  dependent  on  the  existence  of  standardization. 
Once  the  standardization  criteria  has  been  established 
and  systems  have  been  designed  accordingly,  then 
procedural  machinery  must  be  set  in  motion  to  retain 
that  degree  of  standardization.  This  means  control. 

The  publication  of  procedures  to  define  that  which 
is  to  be  controlled  and  the  method  by  which  control  is 
to  be  exercised  is  adequate  for  the  purposes  of  imple¬ 
mentation.  However,  effective  control  is  impossible 
unless  the  necessary  capability  is  provided  to  monitor 
system  activities  and  for  follow-up  action  to  assure 
compliance  with  standardization  and  procedural  policies. 
This  requirement  is  critical  in  the  control  of  standard¬ 
ized  software.  Experience  has  shown  that  the  agency 
charged  with  this  responsibility  must  have  detailed 
knowledge  of  the  standardized  system  elements  and,  as 
a  matter  of  fact,  the  agency  performing  software  con¬ 
trol  must  actually  maintain  the  standardized  elements 
of  the  system. 

This  brings  us  to  the  crux  of  the  requirement  for  an 
“overhead”  facility  and  the  necessity  for  it  to  be  proper¬ 
ly  aligned  organizationally.  There  arc  several  factors 
which  must  be  considered  before  determining  who 
should  be  charged  with  control  responsibility  and 
where  it  should  be  performed.  In  the  first  place,  the 
dynamic  environment  in  which  automated  command  and 
control  systems  must  operate  demands  unfettered  action 
and  reaction  between  system  operators.  This  in  turn 
requires  strong  leadership  to  insure  such  activity  is 
conducted  in  an  organized  manner  thereby  taking  full 
advantage  of  all  opportunities  for  mutual  support.  Sec¬ 
ondly,  control  must  originate  at  the  source.  Each  sys¬ 
tem  operator  has  an  individual  responsibility  to  impose 
strong  control  over  his  system  in  the  interest  of  good 
management  and  to  insure  compliance  with  the  policies 
of  higher  headquarters.  Effective  local  control  is  a  basic 
ingredient  for  success  in  the  Air  Force-wide  manage¬ 
ment  function.  Therefore,  the  overall  control  function 
is  dependent,  to  a  great  extent,  on  the  aggressive  and 
willing  cooperation  of  the  leaders  of  the  various  systems 
involved.  Thirdly,  the  ground  rules  for  control  will  oc¬ 
casionally  make  it  necessary  to  disapprove  a  command 
recommendation.  Obviously  the  controlling  agency  must 
act  with  the  full  support  and  authority  of  the  staff  at 
the  level  where  control  is  to  be  exercised.  Fourth,  an 
agency  assigned  responsibility  for  supervising  standard¬ 
ization  aspects  of  systems  should  not  be  directly  as¬ 


sociated  with  an  operational  system  where  self-inspec¬ 
tion  would  be  necessary  and  where  responsibilities 
would  be  divided  between  the  overall  management  re¬ 
quirement  and  the  local  system  support  requirement. 
Fifth,  the  organization  assigned  the  control  responsibility 
should  be  functionally  aligned  in  a  parallel  manner  to 
the  subordinate  systems.  The  purpose  of  this  is  to  pro¬ 
vide  for  lateral  staff  coordination  within  a  single  func¬ 
tional  area  for  operational  command  and  control  mat¬ 
ters.  Finally,  control  can  only  be  imposed  on  the  basis 
of  respect  for  authority.  More  specifically,  control  must 
be  placed  at  a  level  higher  than  that  of  those  organiza¬ 
tions  being  controlled.  It  cannot  be  conducted  under  a 
theory  that  technical  competence  will  be  the  primary 
qualification  for  the  controlling  agency.  The  proper 
criteria  would  be  to  equip  the  control  function  with 
authority  (through  organizational  alignment)  and  tech¬ 
nical  competence  as  well. 

The  solution  as  to  who  should  control  what  and 
where  it  should  be  performed  is  obvious  to  this  author. 
As  previously  stated,  systems  are  organized  under  and 
assigned  to  the  Director  of  Operations  at  the  various 
organizational  levels  uniformly  throughout  the  Air 
Force.  Therefore,  it  would  be  logical  to  retain  func¬ 
tional  responsibility  of  operational  command  and  con¬ 
trol  matters  under  the  Director  of  Operations  at  Head¬ 
quarters  USAF  where  it  currently  resides  for  the  Air 
Force  Integrated  Command  and  Control  System.  This 
alignment  takes  advantage  of  the  excellent  communica¬ 
tions  which  exist  between  Headquarters  USAF  Com¬ 
mand  Post  and  Air  Force  Major  Commands  and,  more 
important,  keeps  lateral  continuity  in  inter-system  staff 
coordination. 

Under  this  concept,  the  “overhead”  facility  would 
be  a  control  facility  managed  by  operators.  This 
operator  element  would:  Have  an  assigned  person¬ 
nel  capability  to  perform  maintenance  on  standard¬ 
ized  system  software;  provide  technically  oriented  opera¬ 
tional  advice  to  the  staff  of  Headquarters  USAF;  con¬ 
trol  documentation  standards;  establish  operational 
procedures;  and  would  perform  operational  evaluations 
to  determine  the  effectiveness  of  the  Air  Force  systems 
involved.  The  ideal  environment  in  such  a  facility 
would  include  a  technical  support  contingency,  backed 
up  in  depth  by  Air  Force  Systems  Command,  to  ac¬ 
commodate  requirements  for  major  improvements  and 
other  developmental  tasks  which  exceed  the  production 
capability  of  the  system  maintainers.  Placing  the  over¬ 
head  system  directly  under  and  responsive  to  Head¬ 
quarters  USAF7  through  the  Director  of  Operations 
would  provide  the  authority  necessary  for  effective 
control.  With  operationally  oriented  personnel  in  charge 
of  the  facility,  the  cooperation  of  the  Air  Force  major 
commands  involved  would  be  encouraged. 
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To  further  emphasize  this  latter  and  most  important 
point,  it  is  certain  that  operators  will  be  more  prone 
to  assist  and  cooperate  in  the  function  of  control  if 
the  leadership  is  carried  out  by  operators  who  are  “in 
the  same  boat”  and  who  are  directly  associated  with  the 
next  higher  headquarters.  Headquarters  USAF  is  that 
higher  headquarters  in  the  case  of  Air  Force  major 
commands. 

When  an  overhead  facility  is  discussed  in  some  quar¬ 
ters,  the  impression  is  given  that  it  would  be  used 
primarily  for  experimental  and  development  activities 
pertinent  to  current  operational  systems.  It  is  recognized 
that  this  form  of  activity  is  essential  to  evolutionary 
growth  and  improvement  and  is  of  such  importance  that 
it  might  even  be  reasonable  to  invest  in  a  second  over¬ 
head  facility  for  the  specific  purpose  of  experimental 
and  developmental  functions.  However,  if  economical 
considerations  prohibit  this  course  of  action,  then  de¬ 
velopmental  and  experimental  operations  would  have 
to  be  conducted  on  the  control  facility.  Under  this 
situation,  the  overhead  facility  would  primarily  be  a 
maintenance  and  control  facility  with  other  require¬ 
ments  taking  second  place  in  order  of  priority. 

Specific  Areas  for  Standardization 

Within  the  overall  framework  of  the  conceptual  ap¬ 
proach  of  the  Air  Force  in  the  management  of  command 
and  control  systems  and  from  the  system  operators  point 
of  view  presented  herein  on  design  and  development,  a 
strong  case  can  be  made  for  a  realistic  approach  to 
standardization.  Actually  we  are  proposing  a  specific 
plan  of  attack  and  specific  objectives  to  be  used  as  a 
basis  for  development  and  management  of  Air  Force 
headquarters  level  command  and  control  systems. 

From  the  overall  point  of  view,  the  subjeets  pertinent 
for  discussion  are  the  programming  language  and  its 
associated  compiler,  the  retrieval  language,  the  control, 
support  and  utility  program  subsystem,  the  hardware 
configuration,  computer  program  documentation,  and 
general  operating  procedures  and  standards. 

Action  has  been  taken  by  Headquarters  USAF  to 
establish  Jovial  J3  as  the  standard  Air  Force  com¬ 
mand  and  control  programming  language.  This  par¬ 
ticular  language  was  selected  because  it  is  currently  in 
wide  use  and  has  been  successfully  used  over  a  num¬ 
ber  of  years.  It  is  a  solid  base  from  which  evolution  and 
improvement  can  occur,  but  on  a  controlled  basis.  In 
the  near  future  an  Air  Force  Manual  on  Jovial  J3 
and  an  associated  general  compiler  specification  will  be 
coordinated  for  publication.  This  manual  will  establish, 
for  the  first  time,  a  standard  that  is  expected  to  facilitate 
case  in  personnel  transfer  between  systems,  implemen¬ 
tation  of  an  Air  Force -wide  standard  training  pro¬ 


gram  which  the  Air  Force  Training  Command  can 
support,  easier  transfer  of  programs  between  systems 
with  different  makes  of  processors,  and  easier  transition 
to  new  processors  without  total  software  losses.  This 
standard  will  be  particularly  directed  toward  headquar¬ 
ters  level  systems  but  will  be  generally  applied  through¬ 
out  the  Air  Force  with  a  stipulation  that  specific  ap¬ 
proval  will  be  given  for  diversion  from  the  standard  in 
justifiable  cases.  The  primary  objective  is  to  eliminate, 
as  much  as  possible,  unique  language  forms  and  com¬ 
pilers. 

Although  the  above  is  a  statement  of  intent  on  the 
part  of  Headquarters  USAF,  the  primary  concern  of 
the  system  operator  is  not  the  name  of  the  language 
but  rather  one  of  assurance  from  the  policymakers  and 
the  developmental  community  that  (1 )  a  standard  lan¬ 
guage  is  identified  and  associated  with  a  standard  com¬ 
piler,  (2)  changes  to  the  standards  are  centrally  con¬ 
trolled,  and  (3)  the  compiler  (and  its  language)  estab¬ 
lished  as  a  standard  is  determined  to  be  adequate  for 
the  command  and  control  task. 

The  retrieval  language  is  only  beginning  to  receive 
attention.  It  can  be  envisioned  in  the  future  that  par¬ 
ticipating  staff  officers  will  require  direct  access  to  the 
command  and  control  computer  system  through  the 
use  of  communications  consoles.  However,  a  major 
problem  is  confronted  in  training  potential  staff  users 
to  use  a  retrieval  language.  The  staff  offieer  trainee  is 
normally  totally  committed  to  the  pressures  of  his  func¬ 
tional  duties.  He  finds  little  time  for  training  and,  even 
when  given  detailed  training,  his  use  of  the  system  is 
sporadic  in  nature  and  therefore  his  knowledge  of  the 
system  software  capabilities  and  of  the  retrieval  lan¬ 
guage  is  rapidly  lost.  There  is  a  general  feeling  that 
the  concept  can  only  be  successful  over  the  long  period 
if  the  requirement  for  knowledge  and  use  of  the  system 
is  made  a  “way  of  life”  for  staff  personnel.  This  means 
the  introduction  of  the  concept  at  the  pre -com mission 
schools  such  as  the  Air  Force  Academy  and  at  the 
professional  schools  of  the  Air  University.  By  exposure 
to  the  concept  in  an  educational  environment,  the 
possibility  for  permanent  retention  will  be  greatly  en¬ 
hanced.  This  means,  of  course,  that  there  must  be  some 
basic  retrieval  language  standards  used  in  command 
and  control  systems  on  which  the  educational  and  train¬ 
ing  programs  can  be  founded. 

We  have  covered  in  some  detail  the  concept  of  a 
standardized  executive  subsystem  software  complex  in 
previous  paragraphs.  The  advantages  of  this  approach 
are  primarily  based  on  the  requirement  for  economy. 
However,  there  are  inherent  operational  advantages 
which  cannot  be  overlooked.  These  have  also  been 
briefly  covered. 

It  is  the  opinion  of  this  system  operator  that  hard- 
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ware  standardization  can  be  achieved  to  a  very  effective 
degree.  There  will  be  variance  in  the  configuration  be¬ 
tween  individual  systems  in  the  numbers  of  consoles, 
large  panel  displays  and  other  peripheral  devices.  There 
also  will  be  eases  where  a  command  center  is  provided 
more  computing  power  than  the  task  at  hand  is  estimated 
to  require.  However,  these  variances  and  apparent 
possibilities  for  “overbuy”  may  not  be  significant.  The 
basic  software  system  can  be  designed  to  accommodate 
the  varying  numbers  of  displays  and  other  devices  with¬ 
out  impacting  the  standardization  criteria  of  the  software 
system.  The  instances  of  “overbuy”  will  become  in¬ 
significant  when  amortized  over  several  systems.  Fur¬ 
ther,  original  estimates  of  system  requirements  are 
consistently  underestimated.  Finally,  it  is  well  known 
that  hardware  costs  arc  generally  on  a  downward  trend. 
For  example,  the  cost  of  core  memory  was,  at  one  time, 
an  item  which  was  subjected  to  close  scrutiny  by  the 
budget  monitor.  This  is  no  longer  the  case.  The  cost  of 
core  and  other  items  of  hardware  is  considered  in  con¬ 
text  with  overall  objectives  and  costs.  Software  is  the 
big  cost  item  which  is  now  considered  the  area  where 
economy  is  a  necessity.  Economy  is  only  possible 
through  a  concept  of  system  design  which  will  permit 
mutual  and  collective  support  between  command  sys¬ 
tem  operators. 

Computer  system  documentation  is  also  currently 
receiving  attention  at  both  Department  of  Defense  and 
Service  levels.  There  is  a  collective  effort  under  the 
Organization  of  the  Joint  Chiefs  of  Staff  to  establish 
such  standards  for  the  World-Wide  Military  Command 
and  Control  System.  The  Air  Force  is  supplementing 
these  general  standards  with  supporting  procedures 
which  will  permit  effective  control  over  quality,  pro¬ 
duction  and  use.  A  concurrent  Air  Force  effort  is  under 
way  to  automate  the  development  of  program  specifica¬ 
tions.  The  system  is  planned  to  provide  for  automatic 
production  and  updating  of  program  documentation 
tapes.  Consideration  is  being  given  to  having  these  tapes 
forwarded  to  Headquarters  USAF  where  they  can  be 
interpreted  by  a  special  purpose  machine  which  will 
produce  line-,  paragraph-,  page-  and  flow  diagram- 
associated  off-set  masters.  These  masters  could  then 
be  delivered  to  Defense  printing  for  the  production  of 
final  documents.  The  objective,  of  course,  is  to  econ¬ 
omize  in  the  production  of  documents  as  well  as  remove 
the  drudgery  of  flow  diagramming  from  the  computer 
program  specialist. 

Air  Force  operating  procedures  and  standards  for 
headquarters  level  systems  is  an  area  requiring  con¬ 
siderable  attention  and  action.  In  order  to  provide 
needed  guidance,  work  has  begun  on  the  production 
of  several  Air  Force  manuals  and  regulations.  The 
directives  will  establish  operating  standards  and  will 


provide  guidance  on  such  subjects  as  the  management 
of  operational  systems,  general  facility  requirements, 
and  skill,  manning  and  training  requirements.  These 
documents  arc  being  developed  by  the  Air  Staff  with 
the  support  of  the  Electronic  Systems  Division,  Air 
Force  Systems  Command. 

SUMMARY 

The  headquarters  level  command  and  control  system  is 
presently  receiving  concentrated  attention  at  the  policy, 
operational,  staff  management  and  developmental  levels 
throughout  the  Department  of  Defense  and  particularly 
in  the  Air  Force.  It  is  our  opinion  that  these  types  of 
systems  will,  in  the  future,  evolve  from  a  specified  base¬ 
line  rather  than  being  submitted  to  a  complete  develop¬ 
mental  program. 

The  time  is  past  where  operational  activities  in  a 
command  center  will  be  subjected  to,  and  disrupted  by, 
lengthy  automated  system  developmental  activities.  The 
base-line  system  will  be  developed  in  an  “overhead” 
facility.  The  voice  of  the  operator  is  now  being  heard 
and  is  insisting  on  system  development  from  a  “foot  on 
the  ground”  approach.  He  wants  flexible  and  responsive 
tools  on  which  he  can  respond  today  and  build  for 
tomorrow. 

The  Air  Force  must  approach  its  command  and  con¬ 
trol  systems  from  a  perspective  of  a  worldwide,  total 
resource.  It  must  provide  its  component  commanders 
with  systems  that  can  rapidly  respond  to  the  require¬ 
ments  of  their  respective  unified  commands  and  to 
Headquarters  USAF,  but  with  priority  of  emphasis  to 
the  operational  command  channels.  These  systems  must 
also,  however,  be  designed  to  operate  in  mutual  sup¬ 
port  of  one  another  in  the  “air  environment.”  The  need 
for  mutual  support  is  based  on  the  fact  that  there  is  a 
worldwide  similarity  in  air  operations  through  Air  Force 
Doctrine  for  the  employment  of  Air  Power  and  through 
standard  air  operational  policies  and  procedures  which 
unavoidably  effect  command  and  control  operations. 
The  prospect  for  effective  management  of  this  command 
and  control  resource  embraces,  to  a  great  extent,  the 
same  resource  management  principles  which  have  pro¬ 
vided  the  uniforms  we  wear,  the  aircraft  we  fly  and 
policies  under  which  we  function  as  an  Air  Force. 

The  Air  Force  standardization  objectives  which  have 
been  discussed  are  basic  and  must  be  accomplished 
within  the  framework  of  guidance  provided  by  the  Joint 
Chiefs  of  Staff  for  the  World-Wide  Military  Command 
and  Control  System.  Standardization  will  have  for  its 
purposes  the  attainment  of  the  greatest  degree  of 
economy  possible  without  degrading  the  requirements 
of  Air  Force  commanders  (including  those  requirements 
imposed  by  their  respective  unified  commanders  for 
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commands  so  assigned)  and  to  facilitate  ease  in  mutual 
operational  support,  not  only  between  operating  com¬ 
mands,  but  including  Air  Force  support  commands  as 
well. 

It  must  be  made  clear  that  this  approach  to  the 
overall  management  of  the  Air  Force  command  and 
control  resource  must  not  be  confused  with  the  junc¬ 
tions  of  OPERATIONAL  COMMAND  and  RE¬ 
SOURCE  MANAGEMENT  which  are  respectively  per¬ 
formed  by  the  unified  commands  and  the  services.  The 
essential  point  to  remember  is  that,  in  order  for  these 
functions  to  be  performed  with  continuity  and  effec¬ 
tiveness,  the  Air  Force  command  and  control  resource 
(or  system)  must  be  looked  upon  and  managed  as  an 


entity.  The  mission  of  Air  Force  command  and  control 
systems,  both  individually  and  collectively,  is  to  perform 
all  functions  required  of  them  by  appropriate  authority. 
Therefore,  the  entire  Air  Force  command  and  control 
management  apparatus  must  be  oriented  toward  this 
objective. 

The  Air  Force  Integrated  Command  and  Control  Sys¬ 
tem  is  considered  a  tangible  example  which  validates 
the  concepts  and  opinions  expressed  in  this  paper  as  a 
system  operator’s  point  of  view.  Wc  believe  these  con¬ 
cepts  and  opinions  outline  the  future  profile  for  de¬ 
velopment,  operation  and  use  of  Air  Force  Command 
and  Control  Systems  and  possibly  that  of  the  World- 
Wide  Military  Command  and  Control  System. 


A  methodology  for  command  information  system  analysis 


by  John  A.  Evans 
The  MITRE  Corporation 
Bedford,  Massachusetts 


RATIONALE  FOR  DEVELOPING 
A  METHODOLOGY 

The  requirements  translation  gap 

Effective  application  of  computer  support  at  every  level, 
up  and  down,  of  the  command  hierarchy  constitutes 
one  of  the  necessary  first  steps  in  evolving  toward  a 
more  responsive  command  and  control  system.  This  is 
not  a  simple  assertion;  rather,  it  is  an  indication  of  a 
complex  dilemma  that  exists  between  two  communi¬ 
ties:  the  user,  who  is  operations-oriented,  and  the  de¬ 
signer,  who  is  technologically-oriented.  The  situation  is 
further  complicated  by  the  fact  that  these  communi¬ 
ties  have  tended  to  grow  toward  a  more  inextricable 
interrelationship  with  every  application  of  automated 
aids.  Their  experience,  gained  through  several  iterations 
of  the  painful  command  system  acquisition  cycle,  leads 
them  to  generally  agree  that  there  is  an  inadequate 
contextual  understanding  of  what  a  command  informa¬ 
tion  system  is  and  docs,  and  why.  Similarly,  a  common 
understanding  of  the  command  hierarchy  within  which 
an  information  system  is  embedded  has  yet  to  be 
developed. 

The  results  of  these  inadequacies  provide  the  basis 
for  the  dilemma.  Command  information  users,  for 
example,  tend  to  characterize  and  specify  the  type  of 
computer  support  needed  from  a  base  that  is  cither 
too  gross  (we  need  to  plan  faster)  or  too  narrow  (we 
need  a  computer  with  a  16-K  memory  core).  On  the 
other  handi  designers,  in  spite  of  an  intimate  knowl¬ 
edge  of  existing  technology  and  projected  trends,  are 
unable  to  create  and  put  together  relevant  design  ap¬ 
proaches.  Often,  their  system  designs  incorporate  un¬ 
intentional  biases  favoring  one  solution.  Thus,  de¬ 
tailed  hardware/software  specifications  arc  not  mean¬ 
ingfully  related  to  the  command’s  operational  environ¬ 
ment,  functional  activities,  and  problem  areas. 

The  problem  of  applying  computer  technology  was 
precisely  characterized  by  McLucas  (1966),  who 
stated: 


We  have  been  generating  technology  faster  than 
we  have  been  able  to  apply  it.  ...  /  believe  the 
emphasis  in  the  next  20  years  must  go  more  to 
applying  technology  than  to  creating  new  tech¬ 
nology.  .  .  .  The  next  generation  of  technological 
progress  has  to  concentrate  on  systems  analysis 
and  systems  engineering ,  the  putting  together  of 
complex  systems.  .  .  . 

Failure  to  establish  a  common  contextual  basis  for 
uscr/dcsigncr  interaction  has  retarded  effective  as¬ 
similation  of  computer  support  for  command  informa¬ 
tion  systems.  The  resultant  dilemma  created  can  be 
characterized  as  a  requirements  translation  gap.  Hobbs 
and  Craig  ( 1964)  alluded  to  this  gap  when  they  wrote: 
On  the  one  hand ,  there  is  often  an  explicit  de¬ 
scription  of  the  functions  to  be  executed  by  a  num¬ 
ber  of  black  boxes  with  performance  standards. 
On  the  other  hand,  there  is  usually  a  vague ,  if  any, 
description  of  the  operational  concept  and  require¬ 
ments  for  the  overall  system. 

McCarn  (1965),  at  the  Second  Information  System 
Sciences  Congress,  was  more  explicit: 

Often  some  rather  arbitrary  design  criteria  have 
been  specified,  such  as,  “the  system  must  be  able 
to  respond  to  a  query  in  three  minutes ”  (or  three 
hours),  or  “data  in  the  files  will  include  all  ma- 
terial  received  to  within  the  last  two  hours, “  or 
“displays  in  seven  colors  are  required  with  a  posi¬ 
tional  accuracy  of  .05%  of  full  scale.”  Such 
design  criteria  often  pass  for  system  requirements. 
On  the  other  hand,  the  requirement  may  be  based 
on  “the  increasing  flow  of  information  and  the 
compression  in  time  of  the  decision  making  pro¬ 
cess.” 

The  urgent  need  to  resolve  the  gap 

The  nuclear  age  underscores  the  need  to  develop  a 
method  of  selective  response  (Bonington,  1965), 
tailored  to  various  levels  of  escalation,  and  intensifies 
the  importance,  and  the  complexity,  of  providing  a 
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methodology  for  information  systems  analysis  (Mor- 
genstern,  1962).  More  information"'  about  globally 
dispersed  operational  situations  which  is  more  relevant 
to  the  maintenance  of  context  (Thompson  1964a) 
must  be  more  selectively  and  rapidly  accessed,  assessed, 
and  distributed  within  the  command  hierarchy.  At  the 
same  time,  greater  emphasis  must  be  placed  on  using 
the  entire  command  information  structure  to  develop 
meaningful,  multiple  option  plans,  as  opposed  to  using 
it  primarily  for  the  maintenance,  execution,  and  con¬ 
trol  of  a  strategic  integrated  operational  plan  which 
would  be  implemented  during  the  course  of  a  thermo¬ 
nuclear  spasm  response. 

This  must  be  done  to  guard  against  the  danger  of 
uncontrolled  escalation  and  to  permit  a  more  fine¬ 
grained  tailoring  and  deployment  of  limited  U.S.  com¬ 
bat  force  resources.  Implementation  of  a  selective 
response  policy  (Taylor,  1959)  in  the  face  of  the 
threat  of  escalation  requires  the  command  information 
system  network  to  be  less  stress-sensitive  to  overloads 
and  more  rapidly  adaptable  to  the  data  flow  within 
and  between  commands.  It  also  implies  that  military 
officers  directing  or  involved  in  the  staffing  process 
be  allowed  to  regain  the  initiative  and  flexibility  of 
command  they  have  started  to  lose  to  computer-cen¬ 
tered  command  and  control  systems.  Such  systems 
hinder  initiative  because  they  require  the  skills  of  anal¬ 
ysts/programmers  (middlemen)  to  translate  the  gen¬ 
erally  implied  actions  desired  into  specific  man-to- 
eomputer,  computer-to-man  operating  procedures  and 
assessments.  The  potential  danger  created  is  that  the 
middleman  may  unintentionally  provide  a  misleading 
response. 

Flexibility  also  diminishes  because  the  data  base  is 
rigidly  formatted,  and  man/machinc  procedures  are 
rigidly  planned.  In  some  instances,  the  preplanned 
execution  steps  triggered  by  a  specific  query  may  not 
be  correctly  interpreted  by  the  user.  In  order  to  restore 
flexibility  and  initiative,  the  computer’s  attractive 
storage  and  processing  capabilities  must  not  only  be 
retained,  but  further  exploited.  Third-generation  com¬ 
puter  technology  (Licklider  and  Clark,  1962;  Bennett, 
Haines  and  Summers,  1965),  because  of  its  on-line, 
interactive,  and  distributable  features,  offers  potential 
for  restoring  these  characteristics  to  military  officers. 
If  key  staff  members  are  provided  with  greater  on-line 
control  of  easy-to-access  storage  and  processing 
capabilities,  they  will  be  able  to  effect  significant  im¬ 
provements  in  the  management  and  distribution  of 
data  that  connects  operational  forces  and  sensors  with 
the  operational  commanders  who  alert,  deploy  or  com¬ 
mit  combat  resources. 

Effective  assimilation  of  third-generation  computer 
♦Information  as  distinguished  from  data  (See  page  255). 


technology  emphasizes  the  need  for  first  performing 
more  command  analysis  homework  in  order  to  es¬ 
tablish  a  sound  basis  for  improvements  in  command 
information  system  design:  more  explicitly,  the  require¬ 
ments  translation  gap  must  be  closed.  This  basis  can 
be  achieved  by  working  toward  a  methodology  such  as 
the  one  described  in  this  report.  It  is  believed  that  the 
net  effect  of  implementing  such  a  methodology  would 
be  the  realization  of  a  significant  decrease  in  the 
probability  of  misunderstanding  between  user  and  de¬ 
signer,  and  a  significant  increase  in  the  probability  of 
developing  more  relevant  design  approaches.  Through 
application  of  this  methodology,  some  necessary  first- 
steps  in  evolving  toward  a  more  responsive  command 
and  control  system  could  be  initiated. 

The  nature  of  the  proposed  methodology 

Resolution  of  the  user/dcsigner  dilemma  requires  the 
establishment  of  an  integrated,  problem-oriented  meth¬ 
odology  that  encompasses  the  basic  tools  of  systems 
analysis.  Initially,  the  methodology  implemented  would 
provide  a  frame  of  reference  for  the  user  to  help  him 
delineate  more  relevantly  those  operational  and  func¬ 
tional  command  procedures  that  can  be  improved 
through  incorporation  of  computer  support.  As  the 
methodology  evolves,  it  can  be  extended,  integrated, 
and  synthesized  with  other  approaches  to  achieve  its 
ultimate  objective:  the  creation  of  an  analysis  frame¬ 
work  that  stimulates  more  effective  user/designer  inter¬ 
action  through  a  common  contextual  understanding  of 
command  information  systems  and  the  command  hier¬ 
archy. 

Problem  formulation 

A  systems  analysis-oriented  framework,  such  as  the 
methodology  would  provide,  is  needed  by  the  user/ 
designer  communities  for  reassessment  and  future  re¬ 
direction  of  evolutionary  command  information  system 
development.  Many  experienced  designers  in  industrial 
and  military  organizations  now  agree  (Edwards,  1965; 
Armed  Forces  Management,  1963;  and  Institute  for 
Defense  Analyses,  1961)  that  no  problem  formula¬ 
tion,  the  first  six  phases  in  an  operations  research 
process,  has  been  underrated  and  needs  a  great  deal 
more  attention.  E.  S.  Quade  (1964)  characterized  the 
situation  when  he  wrote: 

A  real  pitfall  is  the  failure  to  allocate  and  to  spend 
a  sufficient  share  of  the  total  time  available  decid¬ 
ing  what  the  problem  really  is.  The  problems  faced 
by  the  system  analyst  frequently  belong  to  that 
class  in  which  the  difficulty  lies  more  in  deciding 
what  ought  to  be  done  than  in  deciding  how  to 
do  it.  It  is  a  pitfall  to  give  in  to  the  tendency  to 
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“get  started ”  without  a  lot  of  thought  about  the 
problem. 

(See  also  Churchman,  Ackoff,  and  Arnoff,  1957) 
Inadequate  characterization  of  the  problem  often  leads 
to  a  study  of  the  wrong  problem  or  to  the  preclusion 
of  some  design  alternatives.  This  can  be  avoided  if 
the  problem  is  stated  in  terms  of  functional  needs, 
without  implying  how  it  is  to  be  solved.  The  tools  of 
analysis  that  can  be  derived  through  the  implementa¬ 
tion  of  the  proposed  methodology  can  be  applied  to 
problem  formulation. 

No  matter  how  inadequately  a  problem  is  charac¬ 
terized,  its  meaning  to  a  systems  engineer  who  attempts 
to  design  a  headquarters  command  information  system 
is  translated  into  an  inability  to  answer  certain  specific 
questions,  such  as: 

•  What  docs  the  commander  and  his  staff  do? 

•  What  arc  the  procedures  and  data  used  in  creat¬ 
ing  command  products,  such  as  orders  and  re¬ 
ports? 

•  What  are  the  throughput  rates  of  the  various 
parts  of  the  system? 

Until  a  common  contextual  understanding  of  the 
command  structure  is  acquired  by  the  user/designer 
communities,  questions  such  as: 

•  What  belongs  in  the  data  base? 

•  How  should  it  be  organized? 

•  How  and  in  what  format  will  the  operator  want 
to  recall  processed  information  from  the  data 
base? 

cannot  be  rationally  answered.  What  is  needed,  then, 
has  perhaps  been  most  cogently  stated  by  Major  Gen¬ 
eral  Gould  (1966):  “The  need  for  automation  in  the 
command  and  control  process  has  required  a  new 
discipline — the  science  of  detailed  analysis  of  the  job 
to  be  automated  and  the  precise  definitization  of  how 
the  job  is  to  be  done.” 

Van  Horn  (1964)  was  one  of  the  first  to  recognize 
that  current  limitations  to  implementation  of  computer- 
augmented  command  systems  stem  largely  from  inade¬ 
quate  analysis  of  information  structures,  decision-mak¬ 
ing,  and  management  organization: 

It  is  interesting  to  note  that  the  majority  of  techni¬ 
cal  people  in  the  command  and  control  field  are 
specialists  in  hardware  design ,  while  the  major 
problems  lie  in  determining  information  require¬ 
ments ,  selecting  good  decision  rules,  and  develop¬ 
ing  the  system  to  implement  these  information 
structures  and  decision  rules.  This  discrepancy  may 
well  be  the  most  significant  problem  in  the  field. 
The  need  to  understand  much  more  about  the  cur¬ 
rent  military  information  systems  structure  before  in¬ 
stalling  computers  was  underscored  by  Lewis  (1965) 
when  he  equated  the  “study  of  command”  to  “the  study 


of  the  distribution  and  exercise  of  power.”  He  further 
hypothesized  a  law  of  “Conservation  of  Power”  which 
dictates  that  any  significant  change  in  a  command  capa¬ 
bility  causes  a  change  in  the  distribution  of  power,  and, 
consequently,  “will  be  resisted — by  some.”  A  primary 
and  particularly  important  objective  of  prior  analysis 
is  to  clearly  indicate,  to  both  the  designer  and  user,  the 
subtle  impact  of  various  possible  computer  support 
alternatives  on  the  user's  ability  to  execute  assigned 
missions. 

Purpose  of  the  report 

The  awareness  of  the  requirements  translation  gap 
accompanied  by  a  growing  sense  of  urgency  to  close  it 
indicates  the  need  for  a  better  understanding  of  the 
problem.  The  purpose  of  this  paper  then  is  to: 

•  clarify  the  nature  of  the  gap; 

•  contribute  toward  a  methodology  for  closing  this 
gap  by  characterizing  the  types  of  analyses/ 
products  needed; 

•  make  an  urgent  plea  for: 

1.  synthesizing  and  extending  the  existing  appli¬ 
cable  analysis  effort,  and 

2.  preparing  an  instructional  package,  complete 
with  methodology  and  case  study  experience. 

The  urgency  of  this  plea  stems  from  the  need  to  ac¬ 
celerate  the  systems  analysis  orientation  of  military'  and 
technical  personnel  who  are  currently  involved  in  taking 
next-steps  toward  evolutionary  development  of  specific 
command  systems. 

Scope 

T  his  paper  focuses  on  the  initial  information  systems 
analysis  effort  (sec  Figure  1 )  which  must  be  under¬ 
taken  in  order  to  provide  systems  designers  and  man¬ 
agement  with  a  basic  operational  and  functional  under¬ 
standing  of  the  command,  and  with  a  context  for 
evaluating  and  controlling  the  direction  of  subsequent 
command  system  development.  These  initial  analysis 
activities  are  emphasized  as  opposed  to  those  which 
take  place  “downstream”  (shaded  areas,  Figures  1 
and  3). 


Figure  1 — Simplified  overview  of  command  informalion  sys¬ 
tem  analysis  and  design  process 
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Because  the  report  concentrates  on  a  methodology 
for  command  information  system  analysis,  the  discus¬ 
sion  and  illustrative  material  focus  on  the  information 
flow  related  to  plan  generation/modifieation — a  func¬ 
tion  of  particular  concern  to  component  command  levels 
and  above  (see  Figure  2). 

The  illustrative  material  in  particular  is  drawn  from 
analyses  of  a  Unified/Speeified  command  and  several 
component  command  headquarters  because  commands 
at  these  levels*  critically  need  a  capability  to  help  them 
replan  more  effectively.  Moreover,  it  is  in  this  area 
that  the  potential  offered  by  computer  technology  ap¬ 
pears  to  have  broadest  application:  namely,  to  permit 
a  more  flexible  command  response  to  a  wider  range  of 
contingencies,  tailored  to  various  levels  of  escalation. 

Organization 

The  remainder  of  this  report  is  organized  into  four 
discussion  areas.  The  first  two  clarify  the  nature  of  the 
requirements  translation  gap  by  defining  some  of  the 
terminology  and  by  presenting  an  overall  description 
of  the  command  information  system  design  process. 
Collectively,  these  sections  provide  a  framework  for 
subsequent  discussion.  The  various  types  of  recom¬ 
mended  analytical  efforts  are  identified  Analysis 
Methodology,  while  the  following  section  briefly  evalu¬ 
ates  the  advantages  and  limitations  of  the  approach. 
The  last  section  is  reserved  for  conclusions  and 
recommendations. 


THE  COMMAND  INFORMATION  SYSTEM 
ANALYSIS  PROCESS—  SOME  BASIC 
TERMINOLOGY 

INTRODUCTION 

A  logical  first  step  for  closing  the  translation  gap  is  to 
discuss  some  basic  terminology  and  characterize  those 


*L.  S  Christie  and  M.  G.  Kroger  (1962)  state  that:  " Com¬ 
mand  functions  involve  broad  problems  of  planning,  assessing 
the  capabilities  of  the  command  forces  and  those  of  the  enemy, 
allocating  resources,  alerting  and  committing  the  command 
forces.  These  functions  require  the  gathering  of  large  amounts 
and  many  classes  of  information,  aggregating  the  information 
and  processing  it  lo  enable  the  commander  to  make  knowl¬ 
edgeable,  deliberate  decisions  in  the  context  of  changing  ob¬ 
jectives.  Control  functions,  on  lhe  other  hand,  characteristically 
involve  the  direct  control  of  weapons  in  situations  where,  al¬ 
though  the  volume  is  large,  the  information  can  be  categor¬ 
ized  into  relatively  few  classes,  and  since  the  objectives  are 
definite  and  fixed,  the  problem  is  one  of  directing  action 
toward  the  objectives  through  error  detection  and  correction.  * 


aspects  which  have  a  bearing  on  improved  analysis  and 
design. 

The  command  hierarchy 

Command  information  systems,  already  intricately 
sensitive,  information  capacity  while  maintaining  the 
related^  are  continuously  converging  toward  a  detailed 
and  standardized  relationship  through  the  acquisition  of 
computer  support  subsystems.  Thompson  (1964b) 
presents  a  vivid  discussion  of  how  the  command  hier¬ 
archy  achieves  its  large,  though  increasingly  stress- 
operational  context  of  commanders  and  their  staffs. 
This  aspect  will  not  be  further  mentioned  except  to 
note  its  relevance  in  comprehending  the  information 
structure  of  the  hierarchy. 

Assuming  further  agreement  can  be  reached  on  the 
distinction  which  has  been  made  between  command 
and  control  functions,  it  is  useful  to  comment  on  the 
time-phased  hierarchical  nature  of  this  broad  planning 
(command  emphasis)  and  operations  (control  em¬ 
phasis)  process  (see  Figure  2).  Since  this  report  fo¬ 
cuses  on  command  information  systems,  it  is  con¬ 
cerned  primarily  with  the  analysis  and  characteriza¬ 
tion  of  the  information  flow  related  to  the  generation 
and  modification  of  operational  plans — plans  which 
focus  on  the  deployment  and  employment  of  forces, 
service-directed  logistics  support  to  those  forces,  and 
transportation  movement  support.  More  specifically, 
this  paper  deals  with  the  component  command  levels 
and  above  (see  Figure  2)  because  of  the  emphasis  of 
these  commands  on  longer-range  plan  generation  and 


i 


Figure  2 — Time-phased  hierarchical  nature  of  operational 
planning  and  operations 
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modification  The  emphasis  is  restricted,  however,  to 
the  development  of  the  multiple-option  type  of  lim¬ 
ited  war  planning  required  to  implement  a  flexible 
response  (i.e.,  contingency  planning)  as  opposed  to  the 
emphasis  given  to  a  thermonuclear  spasm  response 
which  is  characterized  by  a  more  centralized,  detailed 
type  of  preplanning. 

Note  that  both  centralized  and  decentralized  infor¬ 
mation  processing  are  desirable,  depending  upon  the 
crisis-related  time  period  because,  as  a  crisis  develops, 
massive  amounts  of  dynamic  intelligence  and  resource 
status  information  must  be  acquired^  synthesized  and 
forwarded  up  through  the  hierarchy  (McCarn,  1965). 
After  employment  operations  begin,  additional  opera¬ 
tions  monitoring  data  must  be  digested.  Plan  modifica¬ 
tion  for  shorter  time  periods  must  be  further  delegated 
in  order  to  avoid  what  Thompson  (1964a)  has  called 
the  “catastrophic  fractionization  of  context,”  caused 
by  distorted  reports  and  time  delays.  Once  the  crisis 
action  operations  have  stabilized  to  some  degree,  a 
tendency  to  reassert  centralized  planning  occurs.  View¬ 
ing  the  command  information  structure  in  this  context, 
it  is  perhaps  easier  to  see  why  more  attention  must  be 
given  to  understanding  the  existing  hierarchical  planning 
and  operations  process  in  order  to  effectively  assimilate 
computer  technology  in  a  wav  which  increases  its 
flexibility  and  responsiveness.  If  this  is  not  accom¬ 
plished,  the  formal  and  informal  staff  to  staff  and 
intrastaff  communication  network,  which  has  served 
well  over  the  years,  will  become  rigid! fied  and  less 
responsive  to  the  command  hierarchy. 

Command  information 

The  primary  reason  for  developing  elaborate  data 
processing  schemes  is  to  supply  the  command  organiza¬ 
tion  with  the  critical  facts  (i.e.,  information)  needed 
to  plan  and  control  operations.  By  conceptually  dis¬ 
tinguishing  data  from  information,  systems  analysts 
can  concentrate  on  mapping  only  the  critical  aspects  of 
the  command  information  structure  and,  in  addition, 
designers  will  have  better  guidance  for  determining  to 
what  part  of  the  data  base  computer  support  can  be 
applied.  Throughout  this  report,  data  refers  to  all  the 
facts  which  could  be  obtained,  while  the  term  infor¬ 
mation  denotes  the  particular  facts  commanders  and 
their  staffs  ought  to  know  in  order  to  address  a  par¬ 
ticular  problem.  Information  is  thus  derived  from  raw 
data,  but  it  has  certain  qualities,  such  as  accuracy, 
timeliness  and  relevance  to  a  particular  situation.  These 
qualities  permit  the  analyst,  designer  or  manager  to 
assess  its  value  (Gregory  and  Van  Horn,  1964)  along 
with  the  cost  of  obtaining  this  information  via  more 
sophisticated  data  processing  techniques. 


Command  information  system 

In  this  report,  it  is  assumed  that  “command  is  a 
function  of  sophisticated  people  in  a  complex  environ¬ 
ment  responding  to  an  ambiguous  situation.”*  Thus,  a 
command  information  system  consists  of  the  people, 
facilities,  equipments  and  procedures  which  enable  the 
commander  and  his  staff  to  interact  in  an  attempt  to 
define  the  ambiguous  situation,  to  respond  with  ap¬ 
propriate  plans,  and  to  assign,  alert,  commit  and  moni¬ 
tor  forces  as  required. 

The  boundaries  (i.e.,  data  input  and  sources,  process¬ 
ing  capabilities,  output  products  and  destinations,  etc.) 
of  a  command  information  system  network  are  fre¬ 
quently  defined  by  the  limits  of  authority  delegated  to 
the  organizations  jointly  charged  with  providing  auto¬ 
mated  support  to  the  command.  Within  a  particular 
command,  the  following  factors  have  tended  to  restrict 
even  more  severely  the  definition  of  the  command  in¬ 
formation  system: 

a.  limited  in-house  systems  analysis  resources  and/ 
or  restricted  freedom  of  action  necessary  for  the 
identification  and  analysis  of  intrastaff  informa¬ 
tion  flow; 

b.  restrictive  command  system  acquisition  proce¬ 
dures  which  have  not  been  modified  to  allow  for 
early  application  of  funding  and  technical  re¬ 
sources  to  focus  on  the  technical  requirements 
translation  problem; 

c.  blurring  of  intrastaff  functional  interfaces  en¬ 
gendered  by  competitive  staff  associated  with 
establishment  of  independent  readiness  centers; 

d.  acquisition  of  computer  suport  training  pack¬ 
ages  (e.g.,  473L  operational  training  capabilities) 
which  have  not  been  tailored  to  a  particular  com¬ 
mand's  set  of  missions; 

e.  inappropriate  level  of  responsibility  and/or  suf¬ 
ficient  commander  interest  in  developing  a  useful 
computer  support  capability;  and 

f.  lack  of  analysis  framework. 

Brief  comments  on  the  last  three  factors  cited  appear 
appropriate. 

The  restriction  imposed  by  d.  presents  a  rather  novel 
situation  in  that  the  arrival  of  a  prepackaged  set  of 
capabilities  could  have  a  major  impact  on  defining 
what  is  perceived  to  be  the  information  requirements  of 
the  command.  Particular  care  must  be  taken  to  ensure 
that  the  advantages  of  accelerating  the  training  of  an 
in-house  staff  do  not  lead  to  a  warped  characterization 
of  the  command's  vital  information  requirements.  A 

^Quotation  attributed  to  J  H  Lewis,  Weapons  System  Evalu¬ 
ation  Group,  IDA;  sec  article  by  N.  P.  Edwards  (1965). 
Moore  (1958)  says,  “Knowledge  of  people  is  ihe  core  of 
I  he  improvement  process  because  changes  can  be  brought 
about  only  through  people." 
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sense  of  urgency  exists  to  define  intrastaff  requirements 
to  guide  the  training  capability  toward  one  which  serves 
and  docs  not  warp  the  command's  mission  execution 
capabilities. 

In  regard  to  c.,  note  that  if  there  is  no  top-level 
support  of  an  intrastaff  users  group  to  encourage  the 
identification  of  command-wide  information  require¬ 
ments,  the  command  system  tends  to  become  synon¬ 
ymous  with  those  inputs  which  somehow  reach  the 
command  and  control  organizational  unit  usually  re¬ 
sponsible  for  maintaining  computer  support  to  the 
command  post.  Similarly,  the  direct  interim  outputs 
of  the  computer  (as  opposed  to  the  command)  arc 
likely  to  be  reviewed  as  command  products.  The  in¬ 
formation  “system"  tends  to  be  exclusively  operations- 
oriented  and  is  apt  to  neglect  important  intelligence, 
planning,  and  logistics  interface  requiremnts. 

One  purpose  of  this  report  is  to  contribute  the  frame¬ 
work  cited  in  f.  to 

°  expedite  the  system  analysis  orientation  of  in- 
house  personnel, 

°  draw'  attention  to  the  required  changes  in  com¬ 
mand  systems  acquisition  procedures,  and 

•  intensify  commander  interest  in  assuming  a  more 
active  role  in  sponsoring  a  broader  in-house  in¬ 


formation  requirements  analysis;  this  must  be 
done  in  order  to  successfully  net  together  the 
soon-to-be  automated  readiness  centers  with  the 
training  packages  in  a  way  which  enhances  the 
command's  mission  execution  effectiveness. 

OVERVIEW  OF  THE  COMMAND  INFORMATION 
SYSTEM  ANALYSIS  AND  DESIGN  PROCESS 

Characterization  of  the  analytical  framework 

The  framework  used  in  this  report  to  characterize 
the  nature  of  the  recommended  command  information 
analysis  effort  (sec  page  259)  is  a  two-stream  require¬ 
ments  analysis  and  design  process  flowchart  (sec  Figure 
3).  The  various  types  of  analyses  and  their  respective 
documentation  products  arc  isolated  from  the  design 
process  and  arc  shown  as  a  part  of  a  parallel  stream 
effort.  Preferably,  this  effort  should  be  performed  at 
the  user’s  site,  and,  to  an  increasing  extent,  by  those 
members  of  the  user's  staff  who  have  been  trained  in 
systems  analysis  approach.  It  should  be  noted,  however, 
that  to  the  extent  these  analyscs/products  do  not  exist 
in  an  updated  form  at  any  given  stage  of  a  command 
system  development,  the  methodological  approach  is 
intended  to  be  flexible  enough  for  application  during 
later  stages  of  the  analysis  and  design  process. 
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Figure  3 — Command  information  system  analysis  and  design 
process 
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The  upper  stream  isolates  those  analytical  activities 
essential  to  closing  the  requirements  translation  gap, 
and  indicates  their  relationship  to  the  rest  of  design 
process.  Analytical  efforts  associated  with  the  upper 
stream  are  usually  more  efficiently  accomplished  on¬ 
site  at  the  using  command,  and,  as  the  depth  and 
breadth  of  the  command’s  technical  competence  ex¬ 
pands,  should  be  conducted,  to  an  increasing  extent, 
with  in-house  technical  resources.  Conversely,  analyti¬ 
cal  and  design  efforts  associated  with  the  lower  stream 
can  be  conducted  at  off-site  locations  in  order  to  con¬ 
serve  and  to  gain  access  to  a  broader  spectrum  of 
specialized  technical  resources. 

On-site  analytical  efforts  should  generate  four  major 
products  (documents): 

•  Operational  Environment  Summary 

•  Function  Execution  Summary 

•  Integrated  Functional  Summary 

•  Command  Analysis  Summary 

Their  primary  purpose  is  to  provide,  in  an  iterative, 
usefully  partitioned  fashion,  the  job-oriented  contextual 
understanding  necessary  for  the  development  of  ap¬ 
plicable  design  concepts  for  the  user,  designer  and 
higher  level  acquisition  management. 

The  major  stages  in  the  evolutionary  acquisition  of 
an  improved  set  of  operational  capabilities  arc  shown 
on  the  lower  part  of  the  diagram  to  indicate  that 
while  this  type  of  analysis  may  be  required  throughout 
the  evolution  of  a  command  information  system,  it  is 
emphasized  during  the  preprogram  definition  stage. 
The  command  might  recognize  a  need  for  increased  re¬ 
sponsiveness  of  its  information  structure  or  might  be¬ 
come  aware  of  a  significant  advance  in  the  technologi¬ 
cal  state-of-the-art,  and,  consequently,  would  initiate 


Hardware 

Software 

•  Compatible  Computer  Families 

•  Fast-Response  Systems,  such  as: 

Real-Time  Systems 

Time -Shared  Systems 

Data  Communications  Systems 

•  Extension  of  Optical  Scanning 

Applications  to  read  hand¬ 
written  numerals 
(eventual  automatic 
handwriting/voice 
recognition) 

•  Cheaper  Mass  Storage  Units 

having  incressed  capacity 

•  New  Data  Communications 

Products  made  economical 
through  use  of  time -shared, 
broadband  channels 

•  Output  Display  Devices  more 

widely  used  for  more  rapid, 
more  flexible  man/machine 
communications 

•  New  Programming  Languages 

•  New  Operating  Systems  being 

released,  intimately  tied  to 
the  new  computers 

•  Multi -Programming  and 

•  Multi-Processing  Techniques 

which  raise  questions  of  data 
protection,  sudlt  trails,  and 
job  accounting 

•  Generalized  File  Processing 

Software  for  converting  file- 
oriented  applications  more 
rapidly 

•  Decision  Table  Processors 

for  shortening  software 
application  development 
and  reprogramming 

•  Job/Function  Applications 

Packages  to  reduce 
programming 

Table  1 — Projected  trends  in  computer  technology 


planning  studies  leading  to  the  development  of  these 
products.  The  associated  design/development/acquisi¬ 
tion  agency  assesses  the  implications  of  these  products, 
draws  on  its  prior  command  systems  design  experience 
and  on  its  knowledge  of  software/hardware  trends 
(such  as  those  shown  in  Table  I),  and,  finally,  creates 
specific  design  concepts  applicable  to  that  command. 

Major  analysis  products 

The  first  product,  called  an  Operational  Environ¬ 
ment  Summary,  provides  the  design  effort  wih  a  basic 
understanding  of  the  operational  environment  within 
which  the  command  system  must  assist  in  the  execu¬ 
tion  of  command  missions  (see  page  259,  Operational 
Environment  Analysis).  The  second  product,  called  a 
Function  Execution  Summary,  provides  the  system  de¬ 
sign  effort  with  documents  containing  more  flow-oricnt- 
ed  data.  These  data  indicate  how  each  major  com¬ 
mand  function  identified  in  the  Operational  Environ¬ 
ment  Summary  is  executed.  T  his  type  of  product  could 
be  developed  for  one  or  more  types  of  missions  and/ 
or  scenarios;  it  should  address  dynamic  impacts  on 
the  information  structure,  caused  by  the  superimposure 
of  new  workload  priorities  during  the  various  crisis- 
related  time  periods. 

When  all  of  the  major  command  functions  have 
been  analyzed  in  this  way,  a  more  detailed,  integrated 
study  is  made  of  interactions  among  these  command 
functions  prior  to  and  during  the  execution  of  assigned 
missions.  The  product  of  this  type  of  analysis  is  called 
the  Integrated  Functional  Summary  (see  page  262, 
Function  Execution  Analysis).  In  addition,  a  Com¬ 
mand  Analysis  Summary  document  should  be  pre¬ 
pared  as  required  to  update  and  summarize,  from  an 
integrated  point  of  view,  the  results  of  previous  infor¬ 
mation  system  analysis  efforts. 

Mission  growth  trends  and  effectiveness  criteria 

In  order  to  avoid  a  common  pitfall  which  involves 
the  application  of  computer  technology  to  an  unex- 
amined,  existing  information  structure,  mission  growth 
trends  and  effectiveness  criteria  are  also  derived  con¬ 
currently.  These  criteria,  and  derived  indices  (mea¬ 
sures)  that  indicate  the  degree  to  which  they  are  satis¬ 
fied,  assist  in  the  assessment  of  present  command 
function  performance,  the  isolation  of  problem  areas, 
and  the  evaluation  of  design  alternatives.  The  results 
of  mission  growth  trend  assessment  and  the  derivation 
of  appropriate  criteria  are  also  generally  useful  in 
guiding  the  systems  analyst  toward  critical  information 
flows  and  the  more  important  problem  areas  amena¬ 
ble  to  data-processing  support.  Appropriate  portions 
of  this  information  are  incorporated  into  the  recom¬ 
mendations  of  other  summary  documents  (see  page 
265,  Derivation  of  Functional  Effectiveness  Criteria). 
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The  requirements  baseline 

Because  the  command  system  is  evolutionary  by 
nature,  a  requirements-oriented  information  system 
analysis  provides  a  vital  and  continuous  source  of 
inputs  for  evaluation  and  control  of  the  design  pro¬ 
cess.  In  order  to  ensure  proper  redirection  of  the  design 
process,  the  requirements  baseline  must  be  maintained. 
This  involves  recognition  and  immediate  assessment  of 
any  development  that  indicates  a  change  in  the  com¬ 
mand’s  current  specification  of  requirements.  In  this 
way,  the  operational  and  system  requirements  can  be 
modified  and  adapted  to  changes  in  the  operational 
environment  and/or  to  the  results  of  increasingly  de¬ 
tailed  analysis/verifieation/design  activity  (see  page 
266,  Maintenance  of  the  Requirements  Baseline). 

System  synthesis 

The  analysis  products  are  then  meshed  with  applica¬ 
ble  design  concepts,  generated  from  the  preliminary 
command  system  design  studies,  to  provide  a  frame¬ 
work  for  the  System  Synthesis  effort.  Essentially,  this 
effort  is  used  to  determine  the  several  ways  in  which 
files,  information  flows,  processing  rules,  and  human 
decision-making  activities  can  be  reorganized  to  en¬ 
hance  the  execution  of  major  command  functions.  A 
starting  point  in  the  System  Synthesis  effort  is  the 
derivation  of  specific  operational  capabilities,*  which 
are  then  ranked  according  to  factors  such  as:  the  de¬ 
gree  of  potential  improvement  offered;  the  command’s 
sense  of  urgency;  and  the  ease  of  incorporating  such 
capabilities  into  a  quick-fix  system.  The  capabilities 
selected  are  assimilated  into  an  integrated,  time-phased 
master  plan,  tailored  for  evolutionary  growth. 

Suggested  design  approaches  consider  the  particular 
characteristics  of  data-proeessing  and  display  tech¬ 
niques  as  well  as  the  decision-making  and  other  capabil¬ 
ities  of  the  commander  and  his  staff.  Other  considera¬ 
tions — the  wide  choice  of  hardware  and  software  tech¬ 
niques,  the  problem  of  optimal  allocation  of  task  re¬ 
sponsibilities  between  man  and  machine — indicate  the 
need  for  conceiving  design  alternatives.  At  least  some 
of  these  alternatives,  along  with  projections  of  their 
impacts  on  the  time,  cost  and  resources  drain,  should 
be  incorporated  as  an  essential  part  of  the  Integrated 
Design  Approach  Plan  (see  Design  Spectrum  Ap¬ 
proach). 

Synopsis  of  downstream  design  process 

The  subsequent  system  design  effort  is  condensed 
into  its  more  essential  aspects  at  the  risk  of  conveying 

♦Specific  operational  capabilities  are  defined  as  the  use  of 
a  subset  of  the  data  base  in  conjunction  with  a  set  of  man/ 
machine  procedures  to  accomplish  a  specific  job,  such  as 
flight  following,  report  generation,  etc. 


an  oversimplified  impression.  At  this  stage,  system 
analysis  concerns  the  identification  and  comparison 
of  alternative  design  approaches  in  order  to  select,  at 
least  tentatively,  preferred  conceptual  designs.  This 
is  done  in  accordance  with  the  operational  and  func¬ 
tional  system  requirements  previously  specified  in  a 
first-pass  fashion. 

Downstream  systems  analysis,  an  iterative  process 
(counterclockwise  circle,  Figure  3),  is  needed  to  elimi¬ 
nate  alternatives  which  arc  economically  inefficient  and 
to  select,  from  those  remaining,  the  one  which  best 
meets  the  objectives  of  performance,  operational  avail¬ 
ability  and  cost.  Trade-offs  among  hardware/software 
systems  parameters,  and  among  cost,  performance,  and 
availability  of  operational  capabilities  are  made  in 
order  to  select  a  preferred  approach.  The  effectiveness 
criteria  derived  earlier  are  critical  to  the  analyst  in 
distinguishing  among  design  alternatives. 

The  analysis  becomes  increasingly  detailed  as  the 
more  promising  hardware  configuration  alternatives  and 
their  associated  software  packages  are  reviewed  by 
designer/user  and  higher  level  command  managements. 
As  a  consensus  develops,  more  sophisticated  systems 
analysis  and  design  verification  techniques  are  applied.* 

Design  spectrum  approach 

A  discussion  of  primarily  independent  design  com¬ 
munity  efforts  which  focus  on  determining  the  appro¬ 
priate  degree  of  automated  support  to  command  infor¬ 
mation  systems  is  outside  the  scope  of  this  report. 
However^  a  brief  synopsis  of  a  proposed  design  spec¬ 
trum  approach  is  provided  because  of  the  vital  contri¬ 
butions  it  can  make  to  alleviating  the  user/designer 
dilemma  and  to  closing  the  requirements  translation 
gap. 

Good  design  judgment  includes  careful  considera¬ 
tion  of  alternative  approaehes/techniques.  A  broad 
base  of  such  alternatives  would  permit  the  designer 
to  select  and  apply  the  right  degree  of  automation  as 
he  converts  user  requirements  into  hardware/software 
configurations.  The  resultant  capability  would  not  only 
be  job-tailored,  but  job-insensitive  as  well;  that  is,  the 
capability  would  be  comparatively  insensitive  to  changes 
in  data  input/output  formats,  loads,  analyses,  etc., 
and  comparatively  adaptable  to  changing  job-specific 
requirements. 

Design  approaches/techniques,  if  ordered  according 
to  increasing  degrees  of  sophistication,  provide  a  spec¬ 
trum  of  design  alternatives.  From  these,  the  designer 


♦For  instance,  the  use  of  automatic  flowcharting  techniques 
such  as  AUTOSTATE,  decision  tables,  etc.  See  Van  Horn 
(1964),  Doughty  (1966),  Evans  (1965),  and  Canning  (1962). 
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can  select  the  one  with  a  level  of  sophistication  suffi¬ 
cient  to  meet 

•  the  more  critical  user  requirements,  and/or 

•  those  requirements  that  are  easiest  to  address, 
and/or 

•  those  user  requirements  that  can  be  satisfied  in 
the  shortest  time  period. 

As  he  eliminates  those  approachcs/tcchniqucs  whose 
levels  of  sophistication  arc  insufficient  for  fulfilling  the 
system’s  operational  and  functional  requirements,  the 
designer  gains  design  modification  insights  not  attain¬ 
able  when  working  from  a  narrower,  one-approach 
design  base.  The  correlation  between  planning  and  op¬ 
erations  effectiveness  (in  the  sense  that  cumulative 
needs  arc  met)  and  increasingly  sophisticated  data  pro¬ 
cessing  approachcs/tcchniqucs  arc  shown  in  Figure  4. 


Figure  4 — Major  factors  affecting  the  correlation  of  increasing¬ 
ly  sophisticated  aids  with  cumulative  needs 

Several  characteristics  inherent  in  the  structure  of 
the  design  spectrum  approach  increase  its  attractive¬ 
ness.  First,  in  applying  it,  the  analyst  cannot  help  but 
consider  the  marginal  benefits  associated  with  each 
level  of  sophistication.  This  is  a  natural  fall-out  of 
thinking  through  the  added  value  and  associated  costs 
of  various  degrees  of  sophistication  in  deriving  a  time- 
phased,  cost/cffectivcncss  design  approach.  Second,  the 
design  spectrum  approach  provides  a  unique  oppor¬ 
tunity  for  gaining  some  useful  and  relevant  insights  for 
the  determination  of  the  most  appropriate  type  of  com¬ 
puter  support  to  be  implemented  ae  various  command 
levels  for  common  functional  problem  areas  (McCarn, 
1965).  Weapon-to-target  resource  allocation,  for  ex¬ 
ample,  is  a  functional  problem  area  common  to  several 
levels  within  the  command  hierarchy.  However,  the 


required  response  characteristics  of  the  automated  aids 
may  vary  for  each  level.  Finally,  the  design  spectrum 
approach,  because  of  its  structure,  permits  consideration 
of  several  design  solutions  to  the  same  functional  prob¬ 
lem,  thereby  providing  a  method  for  implementation 
of  the  quick-fix  capability  often  sought  by  the  user. 

The  design  spectrum  approach  thus  can  provide  a 
step  in  closing  the  requirements  translation  gap;  more¬ 
over,  it  is  a  step  that  can  be  taken  independently  by  the 
design  community.  Use  of  the  design  spectrum  ap¬ 
proach  should  also  stimulate  joint  efforts  for  identifi¬ 
cation  of  subtle  requirements  and  joint  discussions  of 
relevant  trade-offs.  This  collaborative  user/designer 
interaction  could  lead  to  the  selection  of  a  design  ap¬ 
proach  that  embodies  the  appropriate  degree  of  auto¬ 
mation  within  acceptable  pcrformancc/cost/availabil- 
ity  constraints. 

ANALYSIS  METHODOLOGY 

This  section  characterizes,  in  greater  detail,  the  meth¬ 
odological  approach  reviewed  previously.  Though  the 
analysis  and  products  are  interrelated,  they  arc  dis¬ 
cussed  separately  in  order  to  clarify  their  primary 
orientation  and  contribution  to  overall  command  in¬ 
formation  system  analysis. 

Operational  environment  analysis 

The  primary  purpose  of  this  analysis  is  to  provide 
an  initial,  broad  understanding  of  the  operational  en¬ 
vironment  for  a  particular  command's  information 
system.  More  specifically,  the  analysis  focuses  on  the 
command  missions  and  operational  environment,  major 
inter-  and  intra-command  interfaces,  design  constraints, 
the  identification  of  major  command  functions,  and 
recommendations  for  further  study. 

Mission  assessment  and  operational  environment 

Initially,  a  clear-cut  understanding  of  the  command's 
assigned  missions  should  be  achieved,  along  with  a 
broad  conception  of  how  the  command  information 
system  supports  the  execution  of  these  missions  and 
why.  An  assessment  of  which  command  missions  will 
grow  in  importance  and,  consequently,  which  infor¬ 
mation  flows  to  stress,  should  also  be  made.  This 
would  require  an  unsophisticated  but  nevertheless  in¬ 
formed  grasp  of  how  factors,  such  as  the  following, 
impact  on  the  command’s  ability  to  perform  its  mis¬ 
sion: 

a  a  range  of  most  probable  threat  scenarios  and 
types  of  locations  (i.c.,  limited  war  in  Southeast 
Asia) ; 

b  operational  concepts  and  doctrine; 

c  magnitude  and  type  of  force  structure/weapons 
to  be  employed;  and 
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d  allocation  of  responsibilities  among  elements  of 
the  Unified/Specified  command  structure. 

The  type  of  information  available,  e.g.,  planning  studies, 
intelligence  estimates,  contingency  plans,  standard  op¬ 
erating  procedures,  and  exercise  evaluations,  assists 
in  providing  some  basis  for  the  assessment.  Data  col¬ 
lection  and  evaluation  techniques,  including  staff  in¬ 
terviews,  participation  in  command  post,  field  and 
testbed  simulation  exercises,  are  of  value.* 

Inter  command  organizational  environment 

A  second  aspect  of  the  analysis  should  focus  more 
spceifieally  on  identifying  and  characterizing  the  func¬ 
tional  nature  of  the  major  external  command  relation¬ 
ships  and  interfaces.  The  policies  and  procedures  (e.g., 
JCS  Publications)  which  define  the  role  (e.g.,  com¬ 
mand  relationships)  of  each  of  the  interfacing  com¬ 
mands  should  be  understood.  In  addition,  the  major 
input/output  documents  (e.g.,  reports,  plans,  etc.) 
should  be  identified,  and  data  relevant  to  the  input 
and  output  products  exchanged  (Kirby,  1964)  should 
be  characterized  in  terms  sueh  as  quality,  quantity, 
and  timeliness. 

Intra-command  organizational  environment 

The  analysis  of  major  staff  activities  (i.e.,  intelligence 
plans,  operations,  logisties,  and  communications)  should 
be  assessed  (sec  Figure  5)  in  terms  of  type,  quality, 
and  actual  versus  desired  timeliness.  Simultaneously, 
a  first-pass  assessment  can  be  made  to  determine  the 
type  and  extent  of  computer  support  that  should  be 
used  to  enhance  the  effectiveness  of  staff  performance 
as  it  carries  out  its  assigned  responsibilities  and  tasks. 
Decriptions  of  any  computer  capabilitcs  already  in 
existence  within  the  command  (e.g.,  in  support  of  the 
command  post  or  other  readiness  eenters)  should  be  in¬ 
cluded  of  course,  and  should  stress  what  operational 
capabilities  exist,  or  are  planned,  and  why. 

Identification  of  environmental  constraints  and 
requirements 

While  conducting  both  the  inter-  and  intra-command 
environmental  analyses,  certain  input/output  docu¬ 
ments  can  be  identified  as  design  constraints.  This 
identification  should  be  applied  to  those  documents 
whose  content,  timeliness,  etc.,  cannot  be  changed 
because  it  is  not  within  the  realm  of  authority  of  either 

'•'Sophisticated  techniques  such  as  combinatorial  analysis  and 
war-games  simulation  are  not  much  help  at  this  stage  in 
determining  the  sensitivity  of  a  particular  command's  ability 
to  execute  missions  under  a  variety  of  threats,  using  the  varied 
force  structure,  etc.  Communication  with  various  analysts  and 
commanders  who  have  addressed  this  particular  command 
problem  or  held  similar  responsibilities  may,  however,  be 
of  significant  value  at  this  time. 


Figure  5 — Synopsis  of  J-staff  activity  analysis 

the  user  or  the  designer  to  do  so.  However,  the  in¬ 
adequacy  or  tardiness  of  this  information  may  be 
critical  to  a  particular  staff’s  effectiveness.  If  so,  steps 
should  be  taken  to  alert  appropriate  higher  level  or¬ 
ganizations  so  that  problem  areas  beyond  the  scope 
of  the  current  design  activity  can  be  resolved. 

Political  and  economic  factors  should  also  form  part 
of  the  constraint  and  requirements  framework.  Ex¬ 
amples  of  such  considerations  include  a  command’s 
strong  desire  for  a  quick-fix  capability  and/or  the  im¬ 
mediate  need  to  focus  on  a  particular  problem  area 
with  a  limited  amount  of  available  funds  and  technical 
staff  resources. 

Identification  and  characterization  of  major 

command  functions  and  their  interrelationships 

The  key  to  appropriate  follow-on  analysis  and  design 
resides  in  a  good  conceptual  identification  and  char¬ 
acterization  of  major  command  functions.  A  conceptual, 
functionally  oriented  model  (sec  Figure  6)  should  show' 
the  various  command  functions  and  their  interrelation¬ 
ship,  and  should  be  derived  for  various  levels.  Problem 
areas  can  be  identified  and  related  to  task  activities 
within  one  command  function.  Subsequently,  an  assess¬ 
ment  of  their  impacts  can  be  made  on  other  command 
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F  igure  6 — Major  command  functions  and  lhcir  interrelation¬ 
ships 

functions  and,  in  turn,  on  a  particular  mission  as  it 
proceeds  through  various  stages  and  time  periods.  The 
model  also  provides  a  ready  grasp  of  the  relative  im¬ 
portance  of  intrastaff  activities  in  support  of  each 
command  function  as  the  command  state-of-alcrt  in¬ 
tensifies  over  the  various  crisis-related  time  periods. 

The  relative  degree  of  high-priority  (computer) 
support  required  for  mission  execution  during  various 
time  periods  is  shown  in  Figure  7,  which  identifies  the 
major  command  functions  and  typical  outputs,  and 
grossly  indicates  their  interrelationship  and  relative 
importance  as  the  command  state-of-alcrt  intensifies. 
The  effectiveness  of  the  command  and  its  information 
system  is  extremely  sensitive  to  the  efficient  execution 


of  a  number  of  major  command  functions  which  cut 
across  external  and  internal  lines  of  staff  organiza¬ 
tion.  Furthermore,  it  is  dependent  upon  the  integrated 
and  simultaneous  execution  of  those  functions  which 
require  high-priority  support  during  various  stages  of 
command  alert. 

These  can  be  best  exemplified  by  comparing  (sec 
Figure  7)  command  functions  4  and  5  with  2  and  3. 
The  first  set  (4  and  5)  docs  not  require  any  priority 
support  during  a  cold  war  or  routine  statc-of-readincss; 
however,  it  docs  have  a  very  high-priority  support  re¬ 
quirement  as  a  specific  contingency  approaches  and  as 
a  subsequent  plan  is  executed.  The  second  set  (2  and 
3)  has  a  continously  high-priority  support  requirement 
associated  with  its  timely  execution,  regardless  of  the 
command  and  statc-of-alert. 

In  regard  to  operational  plan  generation/modifica¬ 
tion,  note  that,  during  a  routine  state-of-alcrt,  a  high- 
priority  value  is  placed  on  a  number  of  globally  dis¬ 
persed  contingency  plans.  As  a  specific  contingency  or 
a  related  set  of  contingencies  (“hot  plans")  appear  to 
be  developing,  planning  attention  focuses  on  the 
modification  of  specific  crisis  plans. 

Meaningful  priority  support  requires  first  that  more 
become  known  about  the  type  and  volume  of  intra¬ 
staff  information  exchanged  in  support  of  the  command 
functions,  and  next  about  priority  activities  within  a 
specific  command  function.  When  these  requirements 
.arc  known,  data-base  size,  file  structure,  and  the  type 
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Figure  7 — Relation  of  major  command  functions  lo  lhe  need 
for  priority  support 
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of  work  station  configuration  (hardware/software) 
can  be  estimated. 

Problem  characterization  and  recommendations 

One  final  aspect  applies  to  the  Operational  En¬ 
vironment  Analysis;  namely,  that  it  should  characterize 
those  problems  identified  to  date  and  recommend  fur¬ 
ther  areas  for  analysis. 

Functional  execution  analysis 

Function  Execution  Analysis  focuses  first  and  in 
depth  on  the  dynamic  flow  of  information  related  to 
a  particular  command  function,  such  as  the  plan  gen- 
cration/modification  function  throughout,  say,  the 
execution  of  a  command's  augmentation  support  mis¬ 
sion.  Iterative  and  more  refined  analyses  around  those 
activity  sequences  (sub-funetions)  which  relate  to 
major  problem  areas  should  be  subsequently  analyzed. 

Identification  of  critical  information  flows 

The  partitioning,  in  a  conceptually  useful  way,  of 
critical  information  flows  associated  with  each  major 
command  function  is  essential  in  an  overall  analysis 
of  command  structures  (McCarn,  1965;  Gotto,  ctal. 
1962).  In  studying  command  information  systems 
which  emphasize  the  broad  problems  of  planning,  a 
useful  initial  step  is  to  achieve  an  understanding  of  the 
hicrarehial  planning-oriented  information  flows  within 
the  total  command  structure.  With  this  objective  in 
mind,  Figures  8,  9,  and  10  were  prepared  to  show, 
in  several  levels  of  iteration,  the  primary  ingredients  of 


a  Function  Execution  Summary  document — the  main 
product  of  a  Function  Execution  Analysis. 

A  static  representation  of  the  family  of  planning 
documents,  i.e.,  plan  generation/modifieation  inputs 
and  outputs,  was  first  developed  (sec  Figure  8).  The 
sequential  interrelationship  of  the  basic  planning  docu¬ 
ments  among  the  commands  involved  in  the  operational 
plan  generation  process  is  highlighted.  The  flow  of 
documents  from  and  to  superior  and  subordinate  com¬ 
mands  is  depicted,  and  a  key  is  included  to  explain  the 
presentation  of  this  type  of  flow  diagram. 

Next,  a  simplified  (Figure  9)  and  then  a  detailed 
(Figure  10)  representation  of  timc-scquenccd  analyses 
of  the  critical  information  flow  throughout  the  command 
structure  were  developed.  Based  on  the  initial  inter- 
and  intra-analyses  of  the  command  structure,  basic 
milestones  in  the  generation  of  a  contingency  plan  can 
be  established  within  the  command  hierarchy  (see 
Figure  9).  Timing  estimates  for  various  scenarios  and 
force  structures  can  be  obtained,  and  both  routine  and 
flap  generation  times  indicated. 

The  flow  process  shown  in  Figure  10  represents 
a  simplified  version  of  the  flowchart  actually  developed. 
Nevertheless,  it  should  provide  the  reader  with  suf¬ 
ficient  perspective  to  see  that  this  type  of  chart  can 
be  used  by  the  system  designer  to  achieve  a  better 
understanding  of  how  a  computer-augmented  command 
system  capability  could  be  developed  to  serve  the  needs 
of  erisis-oriented  plan  generation/modifieation.  The 
level  of  detail  actually  approached  is  dependent  upon 
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Figure  9 — Planning  time  (illustrative)  of  plan  generation 
process 

such  factors  as  the  overall  importance  of  the  command 
function,  its  growth  potential,  and  time  resources 
available  for  analysis.  Input/output  documents  of  data 
used  by  a  particular  command  staff  (C1NC  X  staff  was 
emphasized  in  Figure  10)  in  performing  activities  or 
making  decisions  related  to  the  development  of  opera¬ 
tional  plans  must  be  described  in  detail. 

The  matrix  approach  to  mapping  the  planning 
process  flow 

A  row  is  devoted  to  each  critical  J-staff  shown  in 
Figure  10.  Similarly,  critical  delay  times  associated 


with  a  specific  stage  in  the  planning  process  can  be 
identified  in  a  column  bounded  by  two  milestones,  and 
can  be  related  to  specific  staff/command  activities 
earmarked  for  reduction.  This  matrix  approach  (Figure 
10)  to  mapping  the  planning  process  flow  enables  the 
analyst  to  expeditiously  isolate  those  command-wide 
(between  columns)  and/or  specific  staff  (individual 
row)  activities  which  are  associated  with  planning 
problem  areas  (c.g.,  cause  of  large  delay  times  and 
inaccuracies). 

The  same  matrix  would  be  useful  in  pinpointing 
the  assistance  a  third-generation  distributed  computer 
capability  could  provide  in  making  the  staffing  process 
more  responsive.  The  critical  flow  path  (heavy  lines 
in  Figure  10)  can  be  studied  to  determine  the  nature  of 
the  personalized  data  base,  the  type  of  work  station 
configuration,  and  the  type  of  data  management 
capability  the  staff  must  use  throughout  the  plan  gen- 
cration/modiftcation  process  in  order  to  make  it  more 
effective.  Each  staff  element  must  maintain  a  sufficient 
overall  context,  which  requires  on-line  interaction  with 
a  dynamically  changing  data  base,  as  it  assists  the 
commander  in  defining  and  reacting  the  ambiguous 
situation. 

From  this  emphasis  (see  Figure  10)  on  using  the 
contextual  representation  as  a  framework  for  more 
detailed  analyses,  time-shared  access  priorities  and 


Figure  10 — Preparation  of  an  augmentation/contingency  plan: 
CINC  X  command  example 


264  Information  System  Science  and  Technology 


software  applications  packages  can  be  derived.  The 
computer  support  which  does  exist  is  not  readily 
accessible  (usually  a  single,  large  computer-complex, 
man/machine  interface)  nor  sufficiently  responsive 
(primarily  off-line  batch  processing). 

Study  of  a  diagram  similar  to  that  shown  in  Figure 
10  revealed  that,  as  planned  modification  time  grows 
increasingly  restricted,  fewer  plan  options  were  able  to 
be  considered  prior  to  the  selection  of  a  tentative  con¬ 
cept  of  operation,  and  further  feasibility  testing  became 
more  difficult.  The  diagram  was  used  to  assist  in  isolat¬ 
ing  the  deployment  feasibility  problem  area  and  in 
characterizing  the  conceptual  design  of  computer-sup¬ 
ported  operational  capability  that  is  generally  useful 
to  a  number  of  commands. 

Identification  of  planning  problem  areas 

WITHIN  A  SPECIFIC  COMMAND 

Once  a  broad  appreciation  of  the  hierarchical  plan¬ 
ning  process  and  its  present  limitations  has  been  at¬ 
tained,  identification/analysis  of  the  information  system 
within  a  particular  command  (i.e.,  CINC  X  and/or 
its  component  commands,  AF  X  or  AR  X)  must  be 
made.  First,  several  levels  of  staff  activity  related  to 
each  of  the  major  command  functions  should  be  se¬ 
quentially  traced  and  characterized  by  a  simplified 
representation  of  the  kind  shown  in  Figure  11.  The 


resources  available  and  a  growing  appreciation  of 
where  the  major  problem  areas  lie  might  initially 
limit  the  Integrated  Functional  Analysis.  Instead,  an 
in-depth  Functional  Analysis  can  be  conducted  and 
focus  on  one  of  the  major  planned  command  func¬ 
tions,  such  as  plan  generation/modification. 

This  kind  of  information  flow  analysis  will  identify 
a  number  of  planning  problem  areas  (e.g.,  force  tailor¬ 
ing,  deployment  feasibility  analysis,  etc.)  which  could 
be  aided  by  data-processing  techniques.*  These  were 
designated  as  problem  areas  because  it  became  clear 
that  critical  planning  effectiveness  criteria,  such  as  re¬ 
sponse  time  and  the  accuracy  of  plan  feasibility  testing, 
were  not  being  satisfied  to  the  extent  desired  by  the 
command.  Problem  areas  of  this  kind  usually  have 
four  characteristics  in  common. 

•  At  present,  they  require  a  significant  amount  of 
time  to  accomplish,  thus  delaying  plan  gencra- 
tion/modification. 

•  They  cause  inaccurate  plans  to  be  prepared, 
resulting  in:  over/under  commitment  of  force 
packages  containing  an  inappropriate  mix  of 
combat  capabilities,  inappropriate  time-phased 

*Once  problem  areas  have  been  synthesized  and  man /machine 
task  allocations  developed  as  part  of  this  System  Synthesis 
effort,  thev  are  referred  to  as  man/machine-oriented  capabili¬ 
ties  (see  Figure  11). 
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arrival  of  force  packages,  and  off-load  base 
saturation,  etc. 

•  They  can  be  resolved,  to  some  extent,  by  auto¬ 
mated  data  retrieval,  storage,  and  processing 
techniques. 

•  They  can  be  resolved,  or  at  leased  lessened,  be¬ 
cause  C1NC  X  design  organizations  have  suf¬ 
ficient  authority  to  take  some  corrective  ac¬ 
tion. 

Identification  of  these  problem  areas  provides  addi¬ 
tional  insight  and  understanding  to  the  systems  designer 
and  indicates  a  range  of  potential  applications  (simple 
to  complex)  which  must  be  considered  in  the  develop¬ 
ment  of  integrated  planning  capabilities.  These  prob¬ 
lem  areas  form  a  key  basis  for  the  determination  of 
the  specific  man/machine  operational  capabilities, 
which  arc  subsequently  derived  (sec  Figure  1  1,  Sys¬ 
tem  Synthesis)  and  included  in  an  Integrated  Design 
Approach  Plan  (sec  Figure  3).  An  integrated  ranking 
of  applicable  problem  areas  (see  page  258,  Systems 
Synthesis)  derived  from  the  preparation  of  other  com¬ 
mand  functions  Execution  Summaries  (see  Figure  3), 
should,  if  possible,  be  performed  prior  to  the  submis¬ 
sion  of  the  Integrated  Design  Approach  Plan. 

This  conceptually  useful,  iteratively  analyzed,  parti¬ 
tioning  process  is  essential  to  the  characterization  of 
junctional  command  information  system  requirements. 
It  is  especially  attractive  because  it  can  be  tailored 
to  the  level  of  analysis  effort  available  and  to  the 
desired  degree  of  improvement  that  the  command  fore¬ 
sees  as  necessary.  This  methodological  approach  should 
contribute  to  the  elimination  of  unbalanced  require¬ 


ments  for  command  information  systems. t 

Derivation  of  junctional  effectiveness  criteria 

During  the  course  of  performing  the  operational  and 
functional  analyses,  which  start  with  a  comprehensive 
understanding  of  the  various  missions  and  growth 
trends,  the  systems  analyst  should  concurrently  derive 
functionally  oriented  effectiveness  criteria  (Jacobs, 
1965).  This  process  is  illustrated  for  the  plan  gencra- 
tion/modification  function  (sec  Figure  12).  fn  es¬ 
sence,  the  process  is  one  of  initially  identifying  major 
problem  areas  which,  upon  reflection,  arc  indicative  of 
major  limitations  in  the  present  planning  process. 
Subsequently,  each  of  these  basic  limitations  must  be 
defined  in  an  operationally  meaningful  sense  to  the 
command  under  study.  Simplified  and  casy-to-obscrvc 
indices  of  criteria  satisfaction,  such  as  time  to  produce 
an  outline  plan  during  routinc/flap  conditions,  arc 
identified.* 

^Hobbs  and  Craig  (1%4)  noted  this  imbalance  in  discussing 
the  nature  of  the  specification  problem:  *'Thc  best  that  can 
be  expected  from  such  a  specification  is  optimized  subsystem 
design  because  of  the  fairly  well  defined  black  box  elements 
of  the  system  However,  some  optimization  might  not  even 

*The  analyst  is  advised  to  initially  focus  on  a  fewr  criteria, 
using  easv-to-observe  indices  of  the  degree  to  which  the 
present  planning  process  is  satisfied.  Downstream  in  ihc  design 
process  (Figure  3,  Refined  Systems  Analysis  and  Verification), 
additional  criteria  can  be  defined  and  more  sophisticated  in¬ 
dices/ measures  of  criteria  satisfaction  can  be  included.  The 
additional  evaluation  comprehensiveness  at  this  stage  is  war¬ 
ranted  during  the  test  of  well-defined,  specific,  potentially  high 
payoff,  man/machine  capabilities.  See  Evans  (1965). 
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Fgurc  12 — Derivation  of  functional  effectiveness  criteria 
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Finally,  the  systems  analyst  must  assess  the  relative 
importance  and  the  interrelationships  of  each  of  the 
criteria.  The  P  in  Figure  12  shows  the  degree  to  which 
a  particular  command's  planning  process  satisfies  sev¬ 
eral  criteria.  The  understanding  of  criteria  interrela¬ 
tionship  is  extremely  critical  in  the  conception  of  an 
improved  planning  process.  For  example,  increasing 
the  satisfaction  of  one  criterion,  such  as  planning 
response  time,  may  conflict  with  the  also  desirable 
increased  satisfaction  of  another  criterion,  such  as  the 
importance  of  considering  an  increased  number  of 
planned  options  before  deciding  to  feasibility-test  a 
particular  option  in  greater  detail.  Lewis  (1964) 
touched  on  this  problem  when  he  said: 

Let  us  consider  a  few  of  the  contributing  criteria 
which  would ,  if  we  knew  how ,  add  up  to  being 
“right."  Let's  take  time.  It  is  always  a  Good 
Thing  to  save  time.  There  are  circumstances  in 
which  it  would  appear  that  being  able  to  do  some¬ 
thing  quickly  is  of  extreme  value.  Unfortunately , 
however,  the  value  of  time  saved  cannot  be  quan¬ 
tified,  even  though  it  may  mean  the  difference 
between  one  catastrophe  and  another.  Although 
one  has  this  impression  of  the  value  of  it,  one 
cannot  say  how  much  it  is  worth,  how  much  should 
be  paid  for  it. 

Thus,  data-proccssing  support  must  be  supplied  to 
the  planning  process  in  such  a  way  as  to  improve, 
overall  net  planning  effectiveness  and  not  just  one 
criterion,  such  as  planning  speed.  The  analyst  must 
make  sure  that  broad  mission  improvement  objectives 
(which  focus  on  an  integrated  improvement  of  all 
command  functions)  arc  satisfied,  and  not  just  plan¬ 
ning  objectives.  A  broad  perspective  of  how  computers 
can  assist  in  all  command  functions  (i.e,  the  com¬ 
mand  mission  in  general)  should  be  determined  from 
similar  analyses  of  other  command  functions. 

Consideration  must  be  given  to  the  additional  follow- 
on  problem  of  communicating  extensive  plan  modifi¬ 
cations  through  subordinate  commands  in  the  hier¬ 
archy  which  have  to  understand  and  implement  this 
changed  guidance  in  the  form  of  operational  orders. 

Maintenance  of  the  requirements  baseline 

The  importance  of  maintaining  a  requirements  base¬ 
line  is  perhaps  best  stressed  by  discussing  the  impact 
of  some  unique  command  system  characteristics.  Any 
subsequent  indication  that  the  established  requirements 
need  to  be  changed  should  be  immediately  assessed 
to  assure  appropriate  redirection  of  the  design  pro¬ 
cess.  Mission  growth,  obsolescence,  and  operational 
constraints  as  affected,  for  instance,  by  shifts  in  the 
allocation  of  contingency  and  general  war  responsibil¬ 
ities  within  the  Unified/Specified  command  structure, 


or  by  revised  estimates  of  enemy  capability  and  inten¬ 
tions,  can  have  a  dominant  influence  on  the  projected 
information  requirements  and  desired  set  of  opera¬ 
tional  capabilities. 

Periodic  undating,  resulting  in  the  issuance  of  a 
Command  Analysis  Summary  (see  Figure  3),  should 
be  conducted  to  reflect  the  impact  of  these  changes 
on  functional  design  and  on  the  alternative  design 
approaches  selected.  In  addition,  the  time,  cost,  and 
performance  data  being  generated  during  the  iterative 
refinement  of  design  alternatives  (counterclockwise 
loop.  Figure  3)  may  also  provide  good  reasons  for 
modifying  official  statements  of  system  requirements 
( e.g .,  design  performance  estimates  were  too  optimis¬ 
tic). 

Impact  of  unique  command  information  system  char¬ 
acteristics  on  the  maintenance  of  the  requirements 
baseline 

Command  information  systems,  unlike  the  more 
commonly  understood  weapons  systems  requirements 
analysis,  have  a  number  of  unique  characteristics 
which  make  their  requirements  analysis  job  not  only 
more  complex  but  of  continuous  importance  to  the 
design  job. 

The  first  characteristic  emphasizes  the  fact  that 
man — the  decision-maker — is  an  integral  part  of  the 
command  system  and  not  simply  a  user,  as  in  the 
case  of  a  weapons  system.  The  job  of  analyzing  human 
decision-making  requires  a  knowledge  of  the  social 
sciences  as  well  as  of  the  physical  sciences,  and  a 
greater  familiarity  with  the  job  being  aided  by  the 
system.  Consequently,  requirements  analysis  is  more 
complex. 

Second,  command  information  systems  cannot  be 
bought  off-line  and  in  batches  because  they  are  an 
integral  part  of  the  “nerve”  center  of  an  operating 
on-line  command.  They  must  be  acquired  in  an  evolu¬ 
tionary  manner;  that  is,  an  increased  capability  must 
be  phased  into  the  existing  system  without  disrupting 
its  present  operational  capability.  Thus,  requirements 
must  be  continually  analyzed  and  the  implications  of 
changed  requirements  immediately  assessed  in  order 
to  effect  efficient  control  of  the  design  process. 

Third,  the  missions  and  capabilities  of  the  many 
other  commands  and  their  command  information  sys¬ 
tems,  which  interface  with  the  prime  command  being 
analyzed,  arc  also  continually  changing.  This  fact 
further  complicates  requirements  analysis  and  neces¬ 
sitates  the  constant  surveillance  of  integration  require¬ 
ments. 

In  short,  the  maintenance  of  the  requirements  base¬ 
line  is  complex  and  of  continuous  importance  through- 
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out  the  operational  life  of  the  command  and  its  com¬ 
mand  information  system. 

EVALUATION  OF  METHODOLOGY 
Advantages 

An  attractive  feature  of  the  methodological  approach 
presented  in  this  report  is  its  broad  applicability  to  all 
phases  of  the  command  system  development  process 
and  to  the  development  of  management  information 
systems.  The  characterization  of  major  command  func¬ 
tions  and  specific  characteristics  of  information  flows 
will,  of  course,  differ  for  various  commands,  depend¬ 
ing  upon  mission  responsibilities.  However,  the  con¬ 
ceptual  framework  provided  and  the  characterization 
of  the  various  types  of  requirements  translation  analy¬ 
ses  should  be  useful  from  the  point  of  view  of  pro¬ 
viding  a  running  start,  especially  to  the  systems  analyst 
with  limited  experience  and/or  familiarity  with  the 
command  hierarchy. 

A  second  important  feature  is  the  demonstration, 
through  illustrative  material,  of  the  iterative,  broad- 
and-general  to  partitioned-and-detailed  analytical  ap¬ 
proach  that  can  be  used  in  the  mapping  of  command 
information  flows.  This  approach  allows  the  systems 
analyst  to  maintain  a  broad  and  balanced  perspective 
prevents  ‘‘catastrophic  fractionization”  of  the  sys¬ 
tems  analyst's  context).  Its  application  should  result 
in  a  balanced  specification  of  requirements,  an  in- 
increased  probability  of  identifying  major  problem  areas 
and,  simultaneously,  improved  assessment  of  their  rela¬ 
tive  importance,  as  well  as  a  more  realistic  derivation 
of  effectiveness  criteria. 

The  approach  is  also  flexible  in  the  sense  that  it 
can  be  tailored  to  the  command’s  improvements  objec¬ 
tives,  as  well  as  to  the  funds  and  qualified  technical 
resources  available.  Several  kinds  of  analysis  products, 
such  as  those  illustrated,  were  actually  used  to  provide 
a  context  for  evaluation  of  a  command  (CINC  X). 
They  also  assisted  in  providing  a  context  for  useful 
conceptual  design  approaches  that  led  to  a  significant 
improvement  in  the  command’s  ability  to  rapidly 
tailor  joint  forces  and  assess  the  feasibility  of  various 
deployment  options  ( before  assigning  and  committing 
a  particular  force  package).  Because  its  value  has 
been  demonstrated  and  proven,  the  methodological  ap¬ 
proach  merits  consideration  for  development  and  fur¬ 
ther  application. 

Limitations 

The  major  limitation  to  this  approach  is  that  it  re¬ 
mains  more  of  a  guide  than  an  easy-to-apply  analysis 
manual  because  intelligent  application  of  the  method¬ 
ology  requires  broad  skills  and  experienced  judg¬ 


ment.*  A  possible  pitfall  to  the  use  of  this  type  of 
approach  is  that  the  analyst  could  become  preoc¬ 
cupied  with  the  flowcharting  per  se ,  leading  to  user/ 
designer  frustration  and  disinterest,  overemphasized 
concentration  on  the  existing  system,  and  to  more 
flowcharting,  which  is  very  sensitive  to  alternative  “if” 
flows  at  a  detailed  level.  This  pitfall  can  be  avoided 
if  the  analyst  concentrates  on  the  following  questions: 

•  why  does  the  command  structure  exist? 

•  where  does  it  need  improvement? 

•  how  can  it  be  improved? 

and  on  the  following  objective:  Emphasis  on  critical 
flows — flows  directly  related  to  identifiable  problem 
areas  and  to  the  actual/ desired  outputs. 

Another  limiting  aspect  is  that  this  paper  only  con¬ 
tains  a  description  of  the  initial  step  toward  the  com¬ 
prehensive  methodology  required;  integration  with  oth¬ 
er  approaches  and  extensive  synthesis  must  be  further 
studied.  The  development  of  a  comprehensive,  teach¬ 
able  methodology  (which  could  expedite  the  effective 
assimilation  of  computer  technology  in  the  command 
structure)  depends  upon  a  broad-to-specific  synthesis 
which  uses  an  historically  based  perspective  in  assimi¬ 
lating  various  updated  ingredients  (see  Figure  13). 
Without  a  proper  historical  perspective,  the  analyst 
cannot  project  changes  in  the  operational  environment 
(which  impact  on  mission  trends)  and  relate  to  them 
the  technology  and  tools  which  will  become  available 
for  problem  identification  and  solution.  As  the  need 
for  methodology  is  recognized  and  its  comprehensive¬ 
ness  increased  anu  taught,  the  probability  of  exploiting 
the  potential  of  a  third-generation  computer  technology 
to  improve  the  command  and  control  structures  will 
be  significantly  increased. 

A  quick  survey  of  the  literature  indicates  that  if  ap¬ 
proaches  to  a  methodological  framework  exist,  it  is 
only  in  the  minds  of  a  relatively  few  elite  analysts. 


*The  nature  of  the  skills  required  and  the  belief  that  they 
can  be  taught  has  been  given  credence  by  others.  Parsons  and 
Perry  (1965),  in  summarizing  Davis  (1964),  feel  that  “though 
not  a  science,  the  type  of  skill  required  involves  some  com¬ 
bination  of  technology  and  an  which  would  require  methods 
of  teaching  more  closely  related  to  generating  synthetic  ex¬ 
perience  than  to  instilling  abstract  knowledge."  McCarn  ( 1965) 
alludes  to  additional  skills  which  are  required:  “Almost  in¬ 
evitably  most  of  the  personnel  operating  command  systems 
believe  they  are  performing  adequately.  The  definition  of  what 
can't  be  done  which  ought  lo  be  done  is  usually  viewed  as 
unwelcome  criticism.  It  is  not  surprising  that  good  functional 
requirements  go  undefined.  The  only  requirements  that  really 
get  well  defined  are  those  where  a  command  wants  to  duplicate 
the  efforts  of  another  command  or  organization  from  which 
it  has  not  received  responsive  service  it  thinks  it  could  pro¬ 
vide  itself.  .  .  .  The  control  and  coordination  of  innovation  in 
a  structure  like  the  cham-of-command  is  a  complex  and  diffi¬ 
cult  problem." 
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Figure  13 — Synthesis  ingredients  required  for  development  of 
a  comprehensive  methodology 

Such  information  must  be  made  available  today  to 
assist  the  military/designer  community  in  evaluating 
how  computers  can  be  used  tomorrow  to  improve  the 
command  information  structure. 

CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions 

The  analysis  methodology  and  the  gap 

The  methodology  discussed  in  this  paper  can  assist  in 
evolving  toward  a  comprehensive  understanding  of  the 
command  and  control  structure  as  it  relates  to  the 
information  system  network.  Development  of  this 
methodology  will  contribute  to  a  better  understanding 
of  the  “job,”  hierarchy,  and  environment,  and  a  better 
characterization  of  the  operational  and  functional  re¬ 
quirements.  This  will,  in  turn,  allow  more  effective  as¬ 
similation  of  computer  technology  in  the  command 
and  control  structure. 

In  so  doing,  the  methodology  can  assist  in  closing 
the  requirements  translation  gap — a  gap  which  has 
produced  vague  operational  concepts  and  resulted  in 
the  use  of  arbitrary  design  criteria.  The  contributions 
toward  providing  a  comprehensive  methodology  as  dis¬ 
cussed  in  this  report  consist  of  an  overall  analysis 


framework,  characterization  of  the  specific  types  of 
analyses  and  products  needed,  and  use  of  illustrative 
materials. 

Command  information  system  network 

Until  the  hierarchical  command  information  system 
network  is  widely  understood  by  the  user/designer 
communities,  and  until  the  impacts  of  important  and 
dynamic  recent  changes  on  the  command  structure 
become  an  integrated  part  of  the  design  and  evaluation 
context,  costly  and  time-consuming  errors  will  continue 
to  be  compounded.*  Some  of  the  problem  areas  re¬ 
quiring  assessment  and  solution,  and  examples  of  their 
impacts  on  the  command  and  control  structure,  are 
listed  below. 

•  Implementation  of  a  selective  response  in  the 
nuclear  age. 

Impact :  more  information,  from  more  sources, 
more  carefully  and  rapidly  assessed, 
in  order  to  support  development/ 
evaluation  of  multiple  options,  more 
rapid  replanning,  etc. 

•  Maintenance  of  global  commitments  which  will 
use,  to  an  increasing  extent,  U.  S. -based  combat 
resources,  tailored  with  greater  precision  and 
deployed  with  greater  speed. 

Impact:  intensified  emphasis  on  accelerated 
staffing,  replanning,  etc. 

•  Recent  changes  in  responsibilities  with  the  Uni¬ 
fied/Specified  command  structure,  including  the 
creation  of  a  new  Unified  command. 

Impact:  revised  mission  trends,  new  informa¬ 
tion  sources  and  reporting  systems, 
etc. 

•  Emphasis  on  reshaping  doctrine  and  tactics  to 
increase  the  effectiveness  and  mobility  of  joint 
operations. 

Impact :  more  complex  planning,  increased 
intercommand  interface  problems,  etc. 

•  Emphasis  on  centralizing  control  as  well  as 
planning  responsibilities,  by-passing,  in  some 
instances,  intermediate  command  levels. 

Impact:  input  source  distortion  and  delays, 
fractionization  of  context,  etc. 

•  Intensified  need  to  respond  swiftly  to  fleeting 
targets  of  opportunity. 

•■The  urgency  associated  with  developing  such  a  methodology 
was,  perhaps,  dramatically  underscored  in  a  recent  article, 
when,  in  discussing  a  specific  command  and  control  system, 
lhc  officer  interviewed  stated  that  its  value  as  a  “deterrent 
power  is  unquestionable,  that  it  will  lake  10  years  and  over 
$400  million  to  add  it  to  the  inventory  can  very  well  be  ques¬ 
tioned."  Cited  as  the  main  problems:  a  lack  of  clear-cut  de¬ 
velopment  responsibility  and  improper  definition  of  user  re¬ 
quirements.  Sec  Armed  Forces  Management  (1966). 
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Impact :  more  rapid  data  summation  and 
more  flexible  staffing  at  field  head¬ 
quarters  level  and  below,  etc. 

The  nature  of  these  problems  must  be  more  fully 
understood  if  the  increased  computer  support  to  higher 
level  commands  and  initial  computer  support  to  field 
headquarters  are  to  be  effective. 

Comprehensive  command  system  requirements 

Specific  command  operational  and  functional  re¬ 
quirements  do  not  exist,  although  they  have  important 
and  subtle  impacts  on  effective  use  of  computer  tech¬ 
nology.  As  a  result,  there  is  a  marked  tendency  to 
gravitate  toward  computer-centered  command  and  con¬ 
trol  systems  which  have  disrupted  the  age-tested  staffing 
process  in  unanticipated  ways.  These  factors  have 
contributed  to  command  frustration,  to  less  than  effec¬ 
tive  use  of  computer  support,  and  to  costly  and  time- 
consuming  redesign  effort.*  Nevertheless,  commands 
continue  to  acquire  larger,  as  well  as  additional,  com¬ 
puters  to  support  individual  staff-oriented  readiness 
centers.  They  are  also  trying  to  assimilate  computer- 
supported  training  capabilities  for  which  information 
requirements  have  not  been  completely  determined. 

Adaptation  of  third-generation  computer  technology 

Third-generation  computer  technology  utilizes  sig¬ 
nificantly  improved  display-oriented,  man/maehine  in¬ 
terface  techniques.  Because  these  computers  can  be 
packaged  into  smaller,  lightweight,  more  rugged  con¬ 
figurations,  they  permit  distribution  of  digital  data  man¬ 
agement,  analysis,  and  reporting  capabilities  to  key  staff 
elements  throughout  all  levels  of  the  command  struc¬ 
ture.  This  type  of  computer  capability  holds  potential 
for  reversing  trend  toward  “catastrophic  fractionization 
of  the  commander’s  context,”  and  for  significantly  im¬ 
proving  the  responsiveness  of  the  command  and  con¬ 
trol  structure — once  the  requirements  translation  gap 
is  closed. 

R  ecom  m  en  da  t  ions 

Development  of  analysis  methodology 

Concerted  efforts  must  be  exerted  to  provide  the 
user/designer/management  communities  with  a  more 
comprehensive  design  evaluation  and  control  context. 
An  analysis  methodology  is  required  to  assist  in  pro¬ 

*The  article  (ibid.)  goes  on  to  say,  "While  there  were  about 
60  .  .  .  liaison  officers  in  the  .  .  .  plant  .  .  .  where  the 
system  was  built,  assembled,  and  tested,  they  could  not  pro¬ 
vide  adequate  information  on  what  the  user  really  wanted. 
Time  and  again  scope  changes  were  allowed  as  .  .  .  ‘require¬ 
ments*  changed  .  .  .  And  each  time  this  happened, *'  said  an 
officer  close  to  the  program,  "it  cost  more  money  and  more 
time." 


viding  an  improved  understanding  of  the  command 
information  system  network  and  in  deriving  meaning¬ 
ful  operational  and  functional  requirements  for  specific 
commands.  This  task  must  be  accomplished  first  if 
we  are  to  design  for  an  effective  assimilation  of  third- 
generation  technology — effective  in  the  sense  that  the 
command  information  system  network  will  be  less  sen¬ 
sitive  to  information  volume  overloading  and  more 
rapidly  adaptable  to  staffing  assignments  at  all  com¬ 
mand  levels. 

An  instructional  package,  which  synthesizes  and 
extends  the  existing  applicable  analysis  effort,  must 
be  prepared.  This  package,  complete  with  method¬ 
ology  and  case  study  experience,  could  be  used  in 
training  qualified  blue-suit  and  civilian  personnel  for 
follow-on,  on-site  responsibilities. 

Design  spectrum  approach  to  developing  insights  for 
the  generation  of  requirements  and  design  modifi¬ 
cation 

A  design  spectrum  approach  to  the  development  of 
a  range  of  data-processing  techniques  which  address 
particular  functional  problem  areas  (McCarn,  1965) 
should  be  undertaken.  By  presenting  users  with  a 
variety  of  increasingly  sophisticated  techniques  for  solv¬ 
ing  common  command  problems  (e.g.,  force  tailoring, 
report  generation,  etc.),  unanticipated  requirements/ 
constraints  can  be  discovered  and  design  modification 
insights  obtained.  Furthermore,  chances  are  increased 
of  developing,  in  a  shorter  time,  an  operational  capabil¬ 
ity  to  address  a  specific  user’s  requirement. 

Since  these  design  spectrum  and  methodological  ap¬ 
proaches  complement  each  other,  their  development 
should  be  carried  out  simultaneously.  The  primary  re¬ 
sponsibility  for  implementing  a  spectrum  approach  lies 
with  the  designer,  while  the  responsibility  for  translat¬ 
ing  system  requirements  belongs  to  the  user.  The  im¬ 
plementation  of  the  design  spectrum  approach,  which 
ean  be  aided  by  an  improved  definition  of  operational 
and  functional  requirements,  will  provide  a  better  basis 
for  user/designer  interaction.  This  joint  effort  will 
expedite  the  conception  and  application  of  useful  opera¬ 
tional  capabilities  necessary  to  the  achievement  of  a 
more  responsive  command  information  system. 
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Introductory  Remarks 

Richard  M.  Longmire 
Research  Analysis  Corporation 
McLean,  Virginia 

Dr.  Fnthoven*  has  said, 

“  . .  .  the  state  of  our  analysis  of  the  effectiveness 
of  large  combat  units  involving  a  mix  of  weapons 
and  mobility  is  still  inadequate. . . .  We  are  quite 
certain  that  the  available  measures  of  effective¬ 
ness  such  as  firepower  index  do  not  account 
properly  for  several  important  attributes  of  such 
forces  .  .  .  if  one  wanted  to  make  a  case  that  it  is 
not  possible  to  measure  the  effectiveness  of  Army 
divisions  in  any  meaningful  way ,  one  doubtless 
could  do  so  convincingly.  But  the  problem  is, 
whether  we  like  a  or  not ,  we  have  to  have  such  a 
measure.  .  .  .  This  measure  is  essential  not  only  to 
permit  realistic  force  comparisons  and  determina¬ 
tion  of  force  requirements ,  but  also  to  allow  as 
to  estimate  the  incremental  contributions  of  new 
weapons  or  new  organizational  concepts < 

“ Can  we  develop  the  kind  of  theory  of  the  ef¬ 
fectiveness  of  Army  divisions  that  is  required ? 

"Among  the  approaches  that  need  to  he  pur¬ 
sued  are  field  exercises  and  tests ,  map  exercises, 
war  games,  and  other  simulations,  and  detailed 
empirical  study  of  the  history  of  war. 

"We  need  studies  directed  not  at  the  derivation 
of  broad  abstractions  like  the  so-called  'principles 
of  war,'  but  at  empirical  propositions  w  ith  precise 
quantitative  content .” 

The  Army,  Navy,  and  Air  Force  appreciate  the 
need  for  “field  exercises  and  tests”  to  obtain  quantita- 

*Dr.  A.  Enthoven.  Assistanl  Secretary  of  Defense  tor  Systems 
Analysis.  "Cost-Effectiveness  Analysis  of  Army  Divisions/’ 
30  March  1965,  Part  I  Proceedings  of  US  Army  Operations  Re¬ 
search  Symposium. 


tive  empirical  data  on  command  control  systems 
which  support  large  operating  forces  like  an  Army 
division. 

The  major  Army  tactical  command  control  data 
system  under  development  is  ADS  At-  (Automatic 
Data  Systems  Within  the  Army  in  the  Field).  ADSAF 
is  to  be  composed  of  three  systems:  1  OS  (Tactical 
Operations  System),  CS3  (Combat  Service  Support 
System)  and  TACFIRE  (Tactical  Fire  Direction 
System).  TOS  will  support  the  intelligence,  opera¬ 
tions,  and  fire  support  coordination  functions.  CS;, 
will  perform  personnel,  administrative,  logistics  and 
other  support  functions.  TACFIRE  is  to  be  used  for 
taetieal  and  technical  control  of  field  artillery. 
ADSAF  is  being  considered  for  employment  at 
echelons  from  battalion  through  Army  level. 

The  Army  has  undertaken  a  four-year  experiment 
in  the  Seventh  Army  environment  in  Germany  to 
determine  the  Army-wide  requirements  for  automa¬ 
tion  of  the  Tactical  Operations  System  In  addition, 
an  extensive  program  of  testing  will  be  conducted  at 
Fort  Hood,  Texas,  to  determine  the  automation  re¬ 
quirements  for  the  Combat  Service  Support  System. 
These  two  major  efforts  follow  many  years  of  Army 
field  tests  with  tactical  data  systems  experimental 
hardware  and  software.  Included  in  the  series  of  tests 
were:  tests,  beginning  in  the  early  1950's,  to  evaluate 
computer  support  of  Held  artillery  functions  leading 
to  the  development  of  the  FADAC  (Field  Artillery 
Digital  Automatic  Computer)  System,  which  is  now 
in  operational  use,  and  to  the  requirements  for  the 
follow-on  TACFIRE  System;  the  First  Intelligence 
Simulative  Test  (FIST),  a  division  level  test  in  1962 
to  evaluate  several  concepts  in  intelligence  data  pro¬ 
cessing;  Exercise  Major  Domo,  the  1964  troop  test 
of  a  Field  Army  level  automated  tactical  operations 
center  concept,  designated  the  AN/MSQ-19. 
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The  present  emphasis  in  Army  field  tests  of  tactical 
data  systems  (ADSAF)  is  on  the  development  of 
user  requirements  in  the  operating  environment  of 
the  user  by  representatives  of  both  the  user  (e.g., 
Seventh  Army  for  TOS)  and  the  developer  (Auto¬ 
matic  Data  Field  Systems  Command).  One  of  the 
conclusions  of  the  1 966  Working  Conference  on  Army 
Tactical  Data  Systems*  was:  “Hardware,  software, 
procedures,  and  people  are  so  interrelated  in  ac¬ 
complishing  a  functional  process  that  it  is  difficult 
to  determine  the  user’s  requirements  for  a  process  in 
any  environment  where  the  interrelationship  cannot 
be  observed.” 

Even  though  the  Army  has  been  deeply  involved 
in  field  tests  for  some  time  it  is  clear  that  much  ad¬ 
ditional  work  is  required  to  develop  an  adequate 
technology  to  support  the  design  for,  conduct  of, 
analysis  of  data  from,  field  tests  of  tactical  data  sys¬ 
tems.  It  is  also  difficult  to  translate  such  field  test 
data  into  requirements  for  tactical  data  systems  which 
can  be  supported  on  a  cost-effectiveness  basis.  The 
Research  Analysis  Corporation  is  providing  the  Army 
support  in  this  regard. 

*The  findings  of  the  Working  Conference  are  being  reviewed  by 
the  Automatic  Data  Field  Systems  Command  and  have  not  as  yet 
been  endorsed  by  the  Army. 


The  Working  Conference  also  concluded,  “The 
major  unsolved  problems  in  the  development  of  mili¬ 
tary  information  systems  appear  to  be  in  the  areas  of 
system  design  technology  and  field  testing.”  It  is 
believed  that  the  extensive  Army  tactical  data  sys¬ 
tem  field  experimentation  now  underway  will  pro¬ 
vide  major  contributions  to  the  development  of  a 
suitable  technology  for  field  experimentation.  Perhaps 
these  contributions  will  be  presented  at  the  next 
Congress. 

The  major  paper  in  this  Chapter  discusses  Air 
Force  field  experiments  and  tests  of  command  con¬ 
trol  systems.  This  paper  by  Walter  Lesiw  on  the 
NORAD  COC  work  provides  a  rather  detailed 
history  of  testing  as  a  part  of  the  command  control 
system  development  process.  He  recommends  a  rep¬ 
resentative  but  separate  facility  for  testing;  high¬ 
lights  the  importance  of  documenting  plans  for  ex¬ 
perimentation;  identifies  the  need  for  development 
of  simulation  technology  early  in  the  program;  states 
the  requirement  for  at  least  one  system  engineer  to 
“trouble  shoot”  during  the  test  to  provide  corrective 
action  as  required,  and  for  a  simulation  supervisor 
to  insure  credibility  of  the  simulation;  and  supports 
the  phased  approach  to  system  evolution,  including 
the  development  of  functional  programs  in  increments 
which  are  later  integrated. 


Field  Experiments  and  system  tests  in 
NORAD  COC  development* 

by  Walter  Lesiw 
The  MITRE  Corporation 
Colorado  Springs,  Colorado 


INTRODUCTION 

This  paper  describes  developmental  activites  as¬ 
sociated  with  design  and  evolution  of  the  Head¬ 
quarters,  North  American  Air  Defense  Command 
Combat  Operations  Center  (NORAD  COC).  The 
Center  is  situated  in  a  hardened  site,  referred  to  as 
the  NORAD  Cheyenne  Mountain  Complex  (NCMC), 
southwest  of  Colorado  Springs,  Colorado.  Of  initial 
concern  in  this  review  of  developmental  events  is 
an  examination  of  the  program  of  experiments,  exer¬ 
cises,  demonstrations,  and  tests  employed  in  achiev¬ 
ing  an  operable  integrated  system.  A  further  objective 
is  to  arrive  at  some  conclusions  regarding  the  utility 
of  exploratory  experiments  and  systematic  testing 
in  evolving  large-scale  information  systems  of  the 
type  discussed. 

General  system  characteristics  and  situational  factors 

affecting  development 

Evolution  of  the  Cheyenne  Mountain  system  was 
appropriately  preceded  by  systematic  analysis  and 
documentation  of  organizational  functions  and  infor¬ 
mation  handling  observed  in  an  antecedent  “manual” 
system.  The  analysis  provided  a  basis  for  preparation 
of  design  specifications  for  a  “new”  system  which 
would  automatically  process,  retrieve,  and  selectively 
display  the  large  volume  of  data  received  at  the  COC. 

Early  planning  for  system  acquisition  included  de¬ 
cisions  to:  establish  an  interim  facility  for  assembling 
the  system  incrementally:  initiate  operations  for  ex¬ 
perimentation  purposes;  and,  prepare  the  system  ele¬ 
ments  for  installation  and  test  operations  at  the  hard 


*The  effort  described  in  this  paper  was  accomplished  under  Con¬ 
tract  number  AH  9(628)2^90,  managed  by  the  Cheyenne  Mountain 
Complex  Management  Office,  Electronic  Systems  Division.  Air 
Force  Systems  Command,  The  views  expressed  here  do  not  neces¬ 
sarily  constitute  a  position  agreed  to  by  the  contracting  agency. 


site.  The  process  was  conceptualized  and  documented 
in  the  form  of  discrete  developmental  steps  (phases), 
including  a  guide  for  conducting  experiments  using 
the  configuration  of  equipment,  programmed  logic, 
and  personnel  assembled  in  an  experimental  facility. 
Eor  convenience  the  original  manual  system  was 
designated  Group  I  (Chart  A),  the  experimental 
facility  Group  II,  and  the  COC  scheduled  for  Chey¬ 
enne  Mountain,  Group  III.  Group  II  was  visualized 
as  an  evolutionary  link  between  I  and  III. 

Equipment  installation,  facility  preparation,  pro¬ 
gram  shakedown,  operator  personnel  indoctrination, 
and  experiment  planning  occupied  the  initial  weeks 
of  the  experimental  phase.  During  the  early  months 
of  experimentation  a  separate  study  of  the  proposed 
Cheyenne  Mountain  Complex  was  undertaken  by 
NORAD  at  the  request  of  the  Office  of  the  Secretary 
of  Defense.  The  study  resulted  in  a  recommendation 
for  a  more  austere  system,  a  compressed  develop¬ 
ment  schedule,  and  the  quickest  possible  transition 
from  Group  I  to  Group  III  operational  capabilities. 
New  program  directives  called  for  an  early  end  to  ex¬ 
perimental  activities  and  the  rapid  preparation  of 
plans  for  implementing  a  program  of  Category  II 
System  Development  Test  and  Evaluation  (Dept. 
A.F.,  1963)  in  Group  II,  to  be  completed  in  Group 
III  following  occupancy. 

As  a  consequence  of  these  program  changes  ex¬ 
perimental  emphasis  gave  way  to  activities  which 
were  to  be  clearly  operationally  oriented,  e.g.,  opera¬ 
tor  and  maintenance  procedures  development  and 
simulated  operational  employment.  The  Group  II 
experimental  facility,  still  fragmented  as  a  system 
since  elements  of  hardware,  programs,  and  personnel 
were  yet  to  be  delivered,  and  after  somewhat  less 
than  a  year  of  experimental  utilization,  became  a 
vehicle  for  operational  development  and  terminal 
tests.  This  new  phase  of  “system  testing”  was  carried 
out  under  formally  prepared  Category  1 1  Test  Plans. 
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Chart  A.  Facilities,  Phases,  Programs1 


Group  I 

Group  II 

Group  III 

NORAD  COC-Soft  Site 

Experimental  Facility 

NORAD  COC-Hard  Site 
(Cheyenne  Mountain 

(Manual  System) 

Complex) 

< - Analysis - *> 

< - Experimentation  &  Demonstration - ** 

Phase  C  Phase  C-l  Phase  D  SOSP1 

Category  II  System  Tests-** 

Phase  E* 

(  Cancell  ed  )  (Transition) 

l-E  2-E  3-E 

Explanatory  Notes: 

1.  Phase  designations  correspond  to  program  package  changes  (Phase  E  in  3  increments) 

2.  SOSP — Simulated  Operational  Support  Plan;  a  stop-gap  measure  during  translation  to  Phase  E. 

3.  Category  II  system  testing  of  all  Phase  E  program  increments  occurred  in  Group  III.  The  final  package, 
3-E,  was  employed  in  Group  III. 


The  developmental  system 

A  system  construct  involving  Group  I,  Group  II 
and  Group  III  as  interacting  elements,  each  having 
an  integrity  of  its  own,  is  useful  for  placing  the  total 
developmental  effort  in  perspective.  Group  I,  the 
existing  soft  site  Combat  Operations  Center  provided 
a  point  of  departure,  a  history  of  precedents  and  pro¬ 
cedures  and  a  philosophy  of  operation  of  fundamental 
importance  to  experimental  utilization  of  Group  II. 
When  merged  with  the  new  physical  and  functional 
capabilities  introduced  in  Group  II,  these  became 
the  basis  for  establishing  concepts  of  operations  and 
plans  for  demonstrations  and  evaluation  of  Group  III. 
In  this  larger  context  it  is  appropriate  to  view  Group 
III  not  as  an  entirely  new  system  but  rather  a  com¬ 
bination  of  new  design  elements  and  formidable  rem¬ 
nants  of  a  previous  configuration  which  had  to  be 
functionally  integrated. 

The  experimental  facility  was  assembled  to  repre¬ 
sent  the  NCOC  rather  than  to  provide  an  operational 
prototype.  Consequently,  the  experimental  command 
post  appeared  as  a  single  dais  instead  of  a  3-tiered 
structure,  while  divisional  areas  interacting  with  the 
command  post  were  simply  clusters  of  display  con¬ 
soles  functionally  analogous  to  divisional  design. 
The  command  post  came  closest  to  duplicating  its 
operational  counterpart  because  it  served  as  a  facility 
mock-up  during  dais  layout  design.  On  the  other  hand. 


communications  installed  in  Group  II  exhibited  least 
fidelity  due  to  delays  in  establishing  network  require¬ 
ments  and  developing  procurement  specifications. 

Experimental  system  environment 

The  experimental  facility  was  arranged  to  represent 
the  working  areas  of  five  functional  organizations 
constituting  the  Combat  Operations  Center.  These 
information  users  were  concerned  with  the  perfor¬ 
mance  of  command  post,  surveillance,  tactics  and 
weapons,  reconstitution  planning,  and  communica¬ 
tions  and  electronics  tasks.  Each  activity  was  manned 
by  one  or  more  operator  positions.  A  simulation 
room  was  established  for  centralized  management 
and  conduct  of  experimental  operations. 

Equipment  complex 

The  data  processing  subsystem  for  the  experimen¬ 
tal  phase  consisted  of  a  Philco  Model  21  1  CDP  with 
a  32K  core  memory  storage  unit,  an  interim  low-den¬ 
sity  magnetic  drum  system,  and  1 2  magnetic  tape  units 
for  on-line  processing.  The  data  display  subsystem 
included  15  display  consoles,  each  having  a  keyboard 
hardcopy  unit.  A  Display  Data  Controller  provided 
interface  between  CDP  and  display  consoles. 

Communications  in  the  form  of  30-button  Call  Di¬ 
rectors  were  installed  at  each  operator  position. 
Five  also  were  located  in  the  simulation  room,  and 
one  at  the  Test  Engineers  position  in  the  computer 
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Figure  1 — Fquipmenl  placement  in  modified  command  post, 
Group  II 


area.  Seven  positions  were  equipped  with  hands-free 
speaker  phones.  Each  position  had  access  to  the  dial 
system,  a  public  address  system,  and  every  other  po¬ 
sition  by  push  button  action.  One  hundred  and  ten 
full-period  internal  circuits  were  employed.  All 
external  source  inputs  were  simulated. 

Midway  through  the  experimental  phase,  a  Philco 
Model  212  CDP  with  improved  core  memory  unit 
reliability  was  installed  to  support  processing  tasks 
not  requiring  the  drum  system.  As  a  consequence, 
processes  for  the  generation  of  simulated  data  and 
reduction  of  recorded  data  ran  two  to  three  times 
faster,  faciliting  a  more  rapid  rate  of  system  testing. 
Shortly  thereafter,  a  high  density  drum  was  installed 
to  operate  with  the  2 1 2  CDP  in  support  of  processing 
tasks  necessitating  drums.  I  his  change  brought  about 
a  significant  reduction  in  operator  request  response 
times  (computer  interrogations)  and  a  man/machine 
interface  which  better  resembled  later  develop¬ 
mental  stages. 

During  initial  experimentation  the  command  post 
was  utilized  as  a  facility  for  mock-up  analysis  of 
dais  display  consoles,  communications  components, 
and  general  layout  of  battle  staff  workspace.  Con¬ 
sequently  it  was  not  immediately  available  for  routine 
experimental  system  operations.  However,  an  interim 
command  post  in  the  form  of  a  three  console  cluster 
was  substituted  briefly.  Recommendations  for  con¬ 
figuration  of  the  Group  III  Command  Post,  based 
upon  the  mock-up  analysis  and  dais  study,  were  in¬ 
corporated  in  Group  111  facility  specifications.  Con¬ 
sole  layout  and  communications  (Figure  I)  for  the 


experimental  Command  Post  tests,  were  also  pat¬ 
terned  after  the  mock-up.  Divisional  placement  for 
experimentation  is  shown  in  Figure  2. 

Two  major  display  subsystems,  the  I  arge  Wall  Dis¬ 


play  for  the  command  post  and  the  closed  circuit 


Figure  2 — Equipment  placement  for  divisional  testing,  Group  IT 
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television  network  for  the  COC,  were  not  available 
during  the  initial  12  months  of  system  experimenta¬ 
tion.  However,  the  importance  of  the  large  display 
to  command  post  and  system  tests,  and  the  feasibi¬ 
lity  of  substituting  a  console  for  large  wall  display 
activity,  made  it  expedient  to  simulate  control  and 
utilization  of  the  wall  display  by  assigning  its  func¬ 
tions  to  one  of  the  display  surfaces  of  the  3-console 
cluster  described  earlier  as  the  interim  command  post. 
Absence  of  color,  reduced  screen  size,  and  console 
presentation  of  the  display  were  the  deviations  in¬ 
troduced  by  the  substitution.  Controls  and  functions 
were  unchanged.  This  adaptability  of  consoles  to 
changes  in  functional  assignment  and  physical  loca¬ 
tion  made  experimental  reconfigurations  of  the  faci¬ 
lity  relatively  simple  and  rapid. 

Figure  3  illustrates  the  Group  II  equipment  com¬ 
plex  essentially  as  prescribed  by  the  NORAD  CMC 
Study  Group.  Among  Study  recommendations  was 
the  directive  that  this  interim  configuration  would 
provide  a  development  agency  and  the  user  with  a 
capability  to  test  equipment,  programs,  and  proce¬ 
dures  in  an  “operational  environment”  prior  to  in¬ 
stallation  of  the  system  in  the  hardened  CMC  facility; 
also,  that  this  would  be  done  in  increments  based  upon 
prescribed  delivery  of  progressive  program  modules. 

Program  system 

System  experimentation  in  Group  11  was  paced  by 
the  development  of  three  computer  program  systems 
—  utility  programs,  operational  programs,  and  sup¬ 
port  programs.  Initial  experimental  plans  were  large¬ 
ly  concerned  with  dcterming  the  operability  of  the 


system,  particularly  in  terms  of  program-computer- 
operator  interactions,  and  with  establishing  a  stable 
test  system  which  could  support  planned  experiments. 
Program  packages  (Chart  A)  were  identified  by  cor¬ 
responding  test  phases— Phase  C,  Phase  C-l,  a  stop¬ 
gap  step  for  changeover  (SOSP),  and  Phase  E. 

Phase  C  programs  included  the  utility  system  used 
in  the  production,  testing,  and  maintenance  of  the 
operational  and  support  programs.  Utility  system 
capabilities  directly  supportive  of  system  testing 
were  those  of  Environment  Generation  used  to  pre¬ 
load  the  data  base  and  control  tables,  and.  System 
Tape  Read-In  to  “start  up"  a  test.  The  operational 
program  system  consisted  of  the  group  of  programs 
providing  capabilities  prescribed  by  operational 
specifications  for  the  Phase  C  system.  Support  pro¬ 
grams  consisted  of  a  general  NORAD  environment 
simulation  subsystem,  simulated  message  analysis 
and  conversion  subsystem,  and  data  reduction  and 
analysis  subsystem.  Of  these  support  subsystems, 
the  first  is  a  group  of  JOVIAL  coded  programs 
esigned  to  automatically  generate  simulated  input 
data  for  the  425L  system  (NORAD  COC)  and  to  op¬ 
erate  in  conjunction  with  the  second.  The  second  is 
designed  to  produce  simulation  input  tapes  compat¬ 
ible  with  the  system  of  operational  programs.  The 
third  subsystem  is  a  group  of  programs  that  provides 
for  data  reduction.  Phase  C  programs  retained  their 
identity  through  succeeding  phases  of  program  evolu¬ 
tion  and  are  represented  in  the  current  system. 

Phase  C-l  programs  were  the  result  of  cancellation 
of  a  Phase  D  system  which  would  have  entailed  elab- 


Figure  3 — Equipment  configuration  (Simplex  System) 
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orate  changes  to  C.  The  alternative  was  an  upgrading 
of  Phase  C  by  the  addition  of  a  new  function  and  a 
new  capability  plus  some  organizational  changes 
which  together  constituted  Phase  C-I.  After  three 
months  of  testing  with  the  C-l  programs,  the  experi¬ 
mental  effort  was  interrupted  by  the  NORAD  Study 
Group's  decision  to  initiate  development  and  test  of 
the  final  Phase  E  program  package  in  three  incre¬ 
ments,  I  E,  2E  and  3E. 

The  first  increment  of  the  Phase  E  program  was 
characterized  by  the  absence  of  a  live  interface 
between  consoles  and  operators  and  an  incomplete 
capacity  for  processing  externally  generated  mes¬ 
sages.  Basic  display  capabilities  were  observable 
through  the  use  of  program-simulated  console  actions. 
Increment  2E  permitted  operator  interaction  with 
the  computer  whereas  computer-generated  outputs 
were  not  available.  Input  processing  of  three  im¬ 
portant  data  categories  were  yet  to  be  included.  The 
final  increment  accounted  for  the  processing  of  all 
specified  message  inputs  and  the  generation  of  re¬ 
quired  outputs  to  recipient  agencies.  A  full  comple¬ 
ment  of  man/machine  interactions  was  also  provided. 
System  tests  were  supported  by  the  3E  program 
through  the  remainder  of  Group  II  operations  (less 
than  5  months). 

By  way  of  contrast,  the  Phase  E  system  of  pro¬ 
grams,  after  redirection  by  the  NORAD  Study  Group, 
differed  in  two  significant  respects  from  the  earlier 
systems.  First,  the  automated  support  of  a  number  of 
COC  functions  was  made  relatively  austere.  Comput¬ 
er  processing  of  several  categories  of  mission-related 
data  was  eliminated  and  plans  for  sophisticated  pro¬ 
gram  interaction  with  an  elaborate  COC  organiza¬ 
tional  function  were  dropped,  although  these  capa¬ 
bilities  had  been  introduced  in  the  Phase  C-l  program 
design.  Secondly,  the  rationale  of  program  sequencing 
and  scheduling  was  revised  to  permit  more  efficient 
utilization  of  central  data  processing  capacity,  provid¬ 
ing  a  more  rapid  response  to  input  demands.  Each 
of  these  differences  from  C-l  had  a  significant  effect 
upon  the  design  and  execution  of  system  tests  that 
followed. 

Manning 

System  experiments  and  tests  were  planned  as  inte¬ 
grated  team  operations.  Military  personnel  were 
assigned  the  responsibility  of  manning  operator 
positions  while  supervision  and  support  of  the  test 
system  was  accomplished  by  contractors.  Program¬ 
ming,  test  design  and  test  implementation  were  also 
contractor  supplied.  The  development  and  application 
of  simulation  technology  fitted  to  continuously  chang¬ 
ing  experimental  and  test  needs  were  contractor  tasks 
with  progressive  participation  by  military  personnel. 


Management  of  the  system  acquisition  and  test  pro¬ 
gram  was  carried  out  by  an  agency  of  the  Air  Force 
Systems  Command*  with  the  close  collaboration  of 
NORAD,  the  ultimate  system  user. 

The  personnel  configuration  defining  operator  posi¬ 
tions  and  console  allocations  underwent  several 
changes  in  the  course  of  system  evolution.  Some  were 
a  function  of  test  experience,  others  were  occasioned 
by  reorganization  directives  originating  within 
NORAD.  These  changes  affected  not  only  the  as¬ 
signment  of  consoles  to  positions  but  the  general 
composition  of  the  COC-command  post,  divisional 
organizations,  and  affiliated  (interfacing)  agencies. 
The  number  of  system  positions  manned  during  ex¬ 
perimental  activities  was  not  limited  to  the  set  of 
consoles  employed.  An  initial  experimental  plan 
called  for  a  team  of  three  operators  manning  a  single 
functional  area,  whereas  the  final  demonstration  of 
system  performance  included  24  operator  positions 
distributed  among  all  elements  of  the  COC  including 
battle  staff  personnel  situated  on  the  command  dais. 
Developmental  procedures 

The  requirement  for  procedures  specification  em¬ 
braces  a  wide  spectrum  of  activities  characterizing 
the  system  development  process.  It  is  submitted  at 
this  point  that  the  task  of  evolving  procedures  for 
operating  and  maintaining  a  newly  configured  system 
is  an  much  a  design  problem  as  the  conception  and 
creation  of  the  equipment  and  programs  otherwise 
defining  the  system.  However,  it  is  recognized  that 
more  often  than  not  procedures  arc  developed  on-the- 
spot  as  an  expedient  to  “get  the  system  to  work." 
Acknowledging  the  fact  that  procedures  are  at  least 
in  part  derivatives  of  the  functional  attributes  of 
the  equipment  and  program  mix,  an  examination  of 
these  was  made  in  order  to  establish  initial  operator 
tasks  at  the  divisional  level  and  subsequently  at  the 
command  post  level. 

Additional  procedures  were  necessary  for  conduct¬ 
ing  system  operations.  Among  these  were  the  exper¬ 
imentation  procedures  necessary  to  achieve  objec¬ 
tives  specified  in  planning  documents,  also  the 
simulation  procedures  for  representing  the  environ¬ 
mental  factors  and  conditions  impinging  upon  the 
system  in  operation.  No  less  important  were  the  pro¬ 
cedures  for  collecting,  recording  and  reducing 
experimental  and  test  data  as  well  as  those  procedures 
for  integrating  interaction  between  the  COC  and  af¬ 
filiated  centers  and  agencies  collocated  in  the  Chey¬ 
enne  Mountain  Complex.  The  conduct  and  manage¬ 
ment  of  the  system  development  program,  in  general, 

’The  implementing  agency  was  referred  lo  variously  as  Detach- 
mcnl  No.  10,  Field  Fesi  Force,  and  Cheyenne  Mouniain  C  omplex 
Management  Office,  all  of  ihe  Flecl ronic  Sysiems  Division,  A  I*  S(\ 
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required  establishment  of  documentation  and  report¬ 
ing  procedures  as  did  the  initial  experiments  and  in¬ 
terim  development  activities. 

Simulation  technology 

Earliest  plans  for  experimentation  included  re¬ 
quirements  for  a  Simulation  Group  and  specifica¬ 
tions  for  “realistic  simulation  of  the  system  environ¬ 
ment,  including  the  insertion  of  impromptu  simulated 
‘events’  as  a  challenge  to  system  operators  during  an 
experiment.”  The  system  of  simulation  was  expected 
to  possess  capabilities  for:  external  inputs  from 
existing  NORAD  systems;  intra-system  inputs  such 
as  simulated  console  switch  actions;  equipment  simu¬ 
lation;  time  compression  and  expansion,  including 
time-marked  magnetic  tapes  for  skipping  blocks  of 
tape  recorded  simulated  events;  representations  of 
threatening  and  stressful  environments,  and  a  full 
range  of  variations  within  and  among  these. 

Relationships  between  experiment  design  and  simu¬ 
lation  techniques  were  considered  in  a  preview  of 
general  methods  for  the  generation  and  utilization 
of  simulated  data  for  Phase  C  experimentation.  Sim¬ 
ulation  prerequisites  to  effective  experiment  design 
were  stated  in  the  form  of  three  sequentially  related 
functions: 

•  Specification  of  the  simulated  environment, 

•  Preparation  of  the  simulation  materials,  and 

•  Control  of  simulation  data  during  the  experi¬ 
ment. 

The  first  of  these  was  expected  to  be  a  part  of  de¬ 
tailed  experimental  plans  in  a  form  sufficient  to  guide 
personnel  responsible  for  producing  the  simulation 
materials  of  the  second.  The  third  function  was  viewed 
as  a  management  problem  for  a  simulation  supervisor. 

These  relatively  simple  conceptions  of  simulation 
were  superseded  after  a  period  of  experimentation 
during  which  progressively  larger  segments  of  the 
system  were  integrated  and  exercised  until  all  func¬ 
tional  organizations  were  participating.  The  result 
was  a  system  approach  to  simulation  problem  con¬ 
cepts  and  implementation  techniques  the  scope  of 
which  amounted  to  a  technology  for  the  support  of 
various  aspects  of  system  development  — design,  ex¬ 
periment,  exercise,  test,  training,  demonstration  and 
evaluation. 

With  all  system  functions  beginning  to  operate  syn¬ 
chronously,  and  simulation  techniques  oriented  to¬ 
ward  performance  generation  on  a  system  level,  it 
was  appropriate  to  construct  a  scenario  representing 
hypothetical  war  conditions  which  would  stress  mis¬ 
sion  performance  capabilities  and  capacities  of  the 
system  in  the  time  period  it  was  scheduled  to  become 
operational.  The  product  was  a  system  test  scenario 
which  contained  the  necessary  attack  elements  and 


information  handling  properties  required  to  initiate 
preparation  of  a  simulation  package  for  a  complete 
system  exercise.  These  conditions  provided  the  frame¬ 
work  for  development  of  the  simulation  technology 
which  was  finally  employed. 

Steps  in  the  design  and  production  of  a  simulation 
problem  now  included: 

•  Definition  of  the  rationale  of  the  problem  sce¬ 
nario; 

•  Compilation  of  the  major  events  — sequential 
event  listing; 

•  Preparation  of  briefings —  events  of  relevance 
preceding  the  exercise; 

•  Detailed  specifications  of  the  simulation  ma¬ 
terials— time  factors,  simulation  aids,  voice 
scripts,  etc.,  and 

•  Problem  production  and  checkout. 

The  specifications  were  used  to  generate  punched 
card  decks  which  in  turn  were  used  to  produce  a 
simulation  tape.  Card  deck  listings  were  also  em¬ 
ployed  to  produce  the  scripts  provided  to  the  voice 
simulators.  Checkout  consisted  of  a  time  and  content 
comparison  of  simulation  tape,  an  initial  conditions 
tape,  the  briefing  materials,  and  the  voice  script,  with 
final  verification  accomplished  by  a  dry-run  of  the 
exercise. 

Problem  packages  from  the  simulation  library  con¬ 
sisted  of  the  input  message  card  decks,  the  voice 
script  card  deck,  and  problem  folders.  The  folders, 
organized  separately  for  the  Chief  Simulator,  the 
simulators,  and  the  observers  (discussed  under  data 
recording)  contained  event  listings,  reference  maps, 
voice  scripts,  background  material,  and  operational 
briefings.  Fourteen  problem  packages  were  eventually 
on  file  in  the  Simulation  Library. 

An  important  aspect  of  the  library  is  a  Master  Card 
File  which  contains,  in  modular  form,  sets  of  message 
input  cards  prepunched  for  variations  of  at  least  10 
important  message  types.  The  card  sets  are  cross- 
indexed  to  identify  correlated  messages  (related  in 
time  or  by  virtue  of  other  common  factors).  The  Mas¬ 
ter  Card  File  allows  rapid  and  efficient  construction 
of  new  problems  by  reducing  the  requirement  for 
scripting  new  message  cards.  Variable  fields  in  the 
standard  messages  of  the  card  file  arc  left  blank  to 
facilitate  application  to  new  problems. 

Key  positions  for  the  control  of  simulation  opera¬ 
tions  were  those  of  the  Chief  Simulator  and  the  Test 
Engineer.  The  Chief  Simulator  position  was  frequent¬ 
ly  occupied  by  the  test  designer  since  his  familiarity 
with  problem  details  and  questions  not  covered  by  the 
simulation  materials  contributed  to  maintenance  of 
exercise  continuity.  The  position  of  Texas  Engineer 
was  established  to  place  an  agent,  familiar  with  test 
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objectives  and  requirements  and  total  system  opera¬ 
tions,  at  the  computer  to  assist  on  questions  of  system 
startup,  program  loops,  exercise  aborts,  recoveries 
from  equipment  failures,  and  compilations  of  test  data. 

Simulation  techniques  were  also  utilized  in  special 
test  situations  other  than  those  involving  complete 
system  operations.  In  equipment  testing  and  in  certain 
computer  program  tests,  teletype  punched-paper  tapes 
were  used.  These  tapes  were  developed  by  recording 
selected  live  data  transmissions  or  by  manual  punch¬ 
ing  of  desired  simulations  of  real  transmissions  on 
paper  tape.  The  technique  permitted  effective  test 
activities  which  lead  to  compatible  interaction  among 
the  local  equipment/program  complex,  communica¬ 
tions  facilities,  and  distant  message  processing  instal¬ 
lations. 

Experimental  and  test  data  recording 

Initial  guides  for  recording  of  experimental  data  in¬ 
cluded  provisions  for  manual  recording  by  assigned 
observers  and  automatic  recording  using  a  variety  of 
devices.  Recording  requirements  were  to  be  specified 
in  detail  by  experimental  and  test  plans  and  would 
cover  data  content,  recording  mode,  recording  dura¬ 
tion,  periodicity  of  records,  and  any  unusual  tech¬ 
niques  likely  to  affect  scheduling  of  experiments. 

Subsequent  plans  specified  a  data  recording  system 
involving  computer  program  techniques,  photographic 
methods,  magnetic  voice  tape  techniques,  and  observ¬ 
er  commentary.  The  computer  program  would  pro¬ 
vide  a  capability  for  recording  tables,  registers  and 
other  content  specified  by  the  experiment  designer. 
In  addition,  recording  at  some  or  all  of  the  following 
points  would  be  obtainable: 

•  Before  or  after  operation  of  some  specified  pro¬ 
gram  or  programs; 

•  At  specified  intervals  during  operation  of  some 
specified  program  or  programs;  and 

•  At  set  intervals  independent  of  program  opera¬ 
tion. 

Magnetic  voice-tapes  were  to  record,  with  time 
marks,  the  following  classes  of  telephone  calls: 

•  Calls  between  any  two  tactical  positions  (opera¬ 
tors); 

•  Calls  between  any  tactical  position  and  any 
agency,  real  or  simulated,  which  is  external  to 
the  System;  and 

•  Calls  by  the  Experiment  Supervisor  via  the 
tactical  telephone  network  to  any  position  or 
station. 

Observer  commentaries,  recorded  manually  at  a 
station  or  operating  position,  would  account  for  sig¬ 
nificant  data  impractical  or  impossible  to  obtain  via 
alternative  techniques.  Instructions  in  the  form  of 
observer  test  procedures  were  developed  which  in¬ 


cluded  orientation  statements,  a  review  of  formatted 
data  collection  sheets,  and  guides  for  reporting  results. 

Recording  techniques  specified  for  system  per¬ 
formance  measurements  during  Category  II  tests  in¬ 
volved  on-line  data  recording  of  three  types: 

•  Hardcopy —  each  output  to  keyboard  hardcopy 
devices  constituted  a  record  available  for 
analysis; 

•  Modular  recording  — each  program  module  re¬ 
corded,  in  fixed  format,  data  regarding  the  tim¬ 
ing  of  its  operator  and  its  communication  with 
its  input  buffers  and  other  programs  modules. 
The  content  of  external  input  messages  and 
operator  actions  was  recordable  by  this  method. 
The  selection  of  modules  for  recording  was 
under  operator  control;  and 

•  Assembly  test  recording  — intended  primarily 
as  a  program  checkout  tool,  assembly  test  re¬ 
cording  provided  a  flexible  method  of  recording 
any  portion  of  the  computer  data  base  at  any 
desired  time.  However,  it  could  not  be  utilized 
while  modular  recording  was  in  process  and  was 
not  intended  as  a  routine  recording  technique. 

In  addition,  system  performance  measurement 
involved  recordings  of  off-line  programs  concerned 
with  generation  of  simulated  data  for  use  with  the 
operational  system  since  these  contained  simulated 
input  traffic  histories.  Also  of  interest  were  line  unit 
recordings  consisting  of  data  flow  at  input/output  line 
units  recorded  via  magnetic  tape  equipment,  teletype 
page  printers,  and  teletype  tape  reperforators.  Rou¬ 
tine  equipment  maintenance  records  assembled  for 
compiling  system  availability  data  also  consituted 
an  element  of  the  recording  regimen. 

Experiment  and  test  plans 

The  series  of  experimentation  and  test  plans  pre¬ 
pared  during  the  period  May  1962-December  1965, 
clearly  reflect  the  discontinuity  of  the  developmental 
program.  However,  the  methods  of  approach  pro¬ 
posed  in  successive  plans  demonstrate  a  strong  thread 
of  consistency  particularly  with  regard  to  the  structur¬ 
ing  and  supervision  of  experiments  and  the  concept 
of  integrating  system  operations  as  a  prerequisite  to 
conducting  meaningful  experiments  and  tests.  Initial 
plans  for  Phase  C,  were  of  an  experimental  character 
in  that  they  emphasized  a  period  of  functional  analysis 
and  system  familiarization  as  well  as  a  time  for  as¬ 
sembling  a  body  of  experimental  data  to  determine 
design  modifications  and  shape  advanced  test  plan¬ 
ning.  The  system,  newly  configured  in  the  Group  II 
facility,  was  viewed  as  a  “breadboard"  or  working 
model  to  support  controlled,  laboratory-like  studies. 

Subsequent  plans  for  Phase  C-I,  proposed  a  con¬ 
tinuation  of  experimental  activities  and  the  beginning 
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of  mission-oriented  system  testing,  as  well  as  mock-up 
and  design  analysis  activities  directed  at  establishing 
a  command  post  which  would  be  integral  with  the 
Phase  C  configuration.  Following  a  brief  period  of 
implementation.  Phase  C-l  plans  were  supplanted 
by  a  proposal  for  a  transitional  period  of  Group  II 
utilization  during  which  the  system  would  be  modified 
to  resemble,  in  performance,  the  configuration  recom¬ 
mended  for  operational  development  by  the  NORAD 
Study  Group. 

Final  plans*  called  for  implementation  of  a  program 
of  Category  II— System  Development  Test  and 
Evaluation  (Dept.  AF,  1963)  that  would  culminate 
in  an  operational  system  which  conformed  to  re¬ 
quirements  established  by  the  NORAD  Study  Group. 
Thirteen  areas  of  testing  were  specified  and  the  pro¬ 
gram  spanned  Group  II  and  Group  III  system  con¬ 
figurations.  The  test  plan  terminated  upon  satis¬ 
factory  demonstration  of  system  performance. 

The  overlapping  sequence  of  plans  and  phases 
(Chart  A)  included: 

•  Phase  C  Experimentation; 

•  Phase  C-l  Experimentation,  Mission-Oriented 
Tests,  and  Mock-up  Analysis; 

•  Simulated  Operational  Support  Tests  and  Com¬ 
mand  Post  Operations; 

•  Category  II  System  Development  Test  and 
Evaluation: 

Group  II  Experimental  Facility, 

Group  III  Cheyenne  Mountain  COC,  and 

•  System  Performance  Demonstration  and  Turn¬ 
over. 

Experiment  and  Test  Objectives 

Objectives  for  each  of  the  phases  represented  in 
the  cited  sequence  were  stated  in  corresponding 
plans.  Basic  objectives  of  the  experimental  program 
were:  to  revise  and  augment  preliminary  positional 
handbooks  and  task  analyses,  on  the  basis  of  con¬ 
trolled  experimentation,  in  order  to  provide  the  de¬ 
signer  with  data  required  in  design  of  the  Group  III 
system;  and,  to  evaluate  system  performance  from  an 
operational  point  of  view  and  determine  whether  the 
design  met  military  requirements. 

A  subsequent  statement  of  Phase  C  objectives 
specified  main  and  subordinate  elements.  The  former 
committed  all  experiments  to  the  task  of  evaluating 
the  Phase  C  “package,”  i.e.,  determine  how  the 
425L  System  (NORAD  COC)  could  be  improved  to 
better  meet  the  requirements  of  the  NORAD  Mission. 


*The  program  of  experimentation  and  tests  was  considered  by  the 
System  Program  Office  (SPO)  to  fall  under  the  general  heading 
of  Category  1 1  testing.  A  plan  to  this  effect  was  drawn  up  in  Febru¬ 
ary  1963,  and  superseded  by  a  subsequent  plan  in  January  1965. 


This  was  to  have  been  realized  by  achieving  subordi¬ 
nate  objectives  of: 

•  Proving  the  capability  of  the  test  bed  (opera¬ 
bility  and  compatibility  of  elements); 

•  Familiarizing  evaluation  personnel  with  the 
Phase  C  system  during  its  assembly  and  check¬ 
out; 

•  Providing  design  information  for  use  in  Group 
III  design  by  analysis  of  the  initial  Phase  C  sys¬ 
tem,  also  new,  and/or  modified  system  designs; 

•  Developing  standard  operating  procedures  in 
coordination  with  the  operator/uscr; 

•  Collecting  performance  data  on  the  elements  of 
the  Phase  C  system; 

•  Building  a  background  of  simulation  capability, 
operator  knowledge  and  experimental  tech¬ 
niques  prior  to  more  sophisticated  levels  of  ex¬ 
perimentation; 

•  Developing  operator  task  descriptions,  person¬ 
nel  evaluation  criteria  and  training  procedures 
for  use  in  later  phases,  and 

•  Developing  detailed  operational  performance 
criteria  for  use  in  Category  II  and  III  testing 
of  later  phases. 

The  terminal  statement  of  Phase  C  objectives  in¬ 
cluded  the  following  priority: 

•  Operate  the  Phase  C  experimental  model; 

•  Accomplish  design  improvements  on  the 
model; 

•  Establish  performance  criteria  based  upon 
operation  of  the  system; 

•  Develop  operating  procedures  in  conjunction 
with  user/operator  personnel; 

•  Develop  techniques  for  Category  Testing; 

•  Develop  training  materials  for  operators  and 
other  personnel,  and 

•  Develop  exercise  and  evaluation  techniques. 

Phase  C-l  was  conceived  as  the  second  stage  of 

developmental  testing  to  be  carried  out  in  the  Group 
II  facility.  It  was  to  involve  not  only  the  assessment 
of  pre-specified  hardware  and  programs  applied  to 
simulated  defense  situations  but  the  immediate  “on 
site”  design  and  test  of  procedures  as  well.  A  goal 
of  this  phase  was  the  determination  of  command  post 
layout,  equipment  needs,  personnel  procedures, 
and  displays  for  Group  III.  Specific  test  objectives 
were  concerned  with  development  of  empirical  data 
describing  Group  II  performance  and  demonstration 
of  mission  handling  capabilities  of  the  COC  con¬ 
figuration  as  represented.  In  decreasing  priority,  the 
specifics  of  interest  were: 

1.  Derivation  and  documentation  of  the  per¬ 
formance  attributes  of  Group  II  by  conducting  tests 
which  permitted  observation  and  description  of  posi- 
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tion  functions,  division  operations  and  command 
post  utilization  of  425L  information  resources  and 
automation  capabilities  in  carrying  out  NORAD 
COC  responsibilities. 

2.  Based  upon  test  experience,  derivation  of  a 
summary  profile  of  system  performance  variables  and, 
under  standardized  test  conditions,  collection  of 
sample  measures  of  performance  in  order  to  develop 
quantitative  criteria  for  performance  assessment  and 
mission-oriented  demonstrations. 

3.  Demonstration  and  assessment  of  system  execu¬ 
tion  of  NORAD  mission  responsibilities  employing 
the  C-l,  Group  11,  configuration  under  representative 
conditions  of  COC  operation. 

4.  Specification  and  documentation  of  operational 
procedures  for  C-l  Group  II. 

5.  Design  detailing:  Identification  of  design  prob¬ 
lems  at  any  level  of  system  operation  and  recom¬ 
mendations  for  remedial  measures  (hardware  and 
facility  modification,  procedural  and  personnel 
changes,  program  and  document  alterations,  et.al.). 

6.  Preparation  of  a  test  guide  for  Group  111  testing. 

Five  of  the  objectives  were  augmented  with 
matrices  blocking  out  the  design  approach.  Each 
matrix  represented  a  generalized  data  collection  form 
containing  grouped  variables,  design  conditions, 
special  requirements,  and  probable  observations. 
Command  post  tests,  on  the  other  hand,  were  ex¬ 
pected  to  involve  more  than  experimental  analysis 
and  test,  e.g.,  mock-ups,  command  dais  rearrange¬ 
ments,  and  display  innovations.  Consequently  ob¬ 
jectives  for  these  tests  were  separately  documented 
with  the  prescription  to: 

1 .  Establish  a  basis  for  command  post  test  opera¬ 
tions  by  specifying  techniques  and  procedures  to  be 
employed  by  each  battle  staff  position  in  performing 
mission  tasks  using  Group  II  capabilities  — displays, 
machine  stored  data,  programmed  data  handling, 
simulated  problems,  and  divisional  resources. 

2.  Specify  the  characteristics  of  command  essential 
displays  and  develop  procedures  for  their  preparation 
and  presentation  under  varying  conditions  of  system 
operations. 

3.  Demonstrate  integrated  command  post  opera¬ 
tions  involving  full  command  staff  participation,  in¬ 
teraction  with  lower  echelons,  supporting  activities 
and  eternal  agencies,  under  varying  defense  conditions 
based  upon  performance  data  obtained  under  1  and  2. 

4.  Assess  the  adequacy  of  Group  11  communica¬ 
tions— network  and  end  instruments  — for  supporting 
command  post  operations  during  test  (requirements 
and  preliminary  assessment  criteria  were  supplied). 

5.  Determine  detail  design  requirements  including 


modification  implications  based  upon  command  post 
test  observations  and  analysis. 

These  objectives  represented  a  revision  to  a  previ¬ 
ous  list  which  addressed  Phase  C-l.  The  change  en¬ 
tailed  an  initial  response  to  the  redirection  proposed 
by  the  CMC  Task  Force  implying  a  shorter,  intense 
test  effort  with  emphasis  upon  operational  proce¬ 
dures  development  and  practice  in  system  utilization 
so  as  to  facilitate  transition  from  a  Group  II  test 
complex  to  demonstrated  operational  capability  in 
the  hard  site. 

The  Simulated  Operational  Support  Plan,  intended 
to  fill  the  gap  between  Phase  C-l  experimentation/ 
tests  and  the  initiation  of  Phase  E  tests  under  a  formal 
Category  11  test  plan,  proposed  “simulation”  of  the 
Phase  E  programs  as  well  as  reconfiguration  of  the 
Group  11  test  system  after  the  hardware  and  organi¬ 
zational  concepts  recommended  by  the  CMC  Task 
Force.  Primary  objectives  were  concerned  with: 

•  Providing  a  means  for  implementing  the  com¬ 
mand  post  development  program: 

•  Providing  an  environment  for  operator  proce¬ 
dure  development  for  application  to  Phase  E; 

•  Developing  a  stable  system  configuration, 
describing  it,  and  documenting  the  correspond¬ 
ing  operator  procedures,  and 

•  Developing  a  demonstration  capability  that 
would  provide  basic  system  introductions  to 
user/operator  personnel. 

Secondary  objectives  for  SOSP  were  to: 

•  Continue  development  of  new  simulation  tech¬ 
niques  that  would  be  applicable  to  Phase  E 
problem  design;  also,  explore  various  techniques 
in  supplementary  simulation; 

•  Establish  observer  criteria  and  data  acquisition 
forms  suitable  to  Phase  E  testing,  and 

•  Establish  a  convenient  reference  source  for 
simulation  data. 

The  Category  11  Test  Plan,  prepared  under  Air 
Force  Regulation  80-14,  constituted  the  most  con¬ 
ventional  approach  to  testing  taken  during  the  sys¬ 
tem  development  process.  It  accounted  for  the 
terminal  phase  of  testing  although  covering  a  greater 
interval  of  time  than  all  preceding  experimentation 
and  test  activities  combined.  Group  II  and  Group  III 
tests  were  included  as  well  as  the  three  increments 
of  the  Phase  F  program  package. 

The  prime  purpose  of  the  plan  was  to  “ascertain 
that  the  NORAD  COC  operational  functions  and 
characteristics  are  substantially  in  accordance  with 
the  requirements  specified  in  the  CMC  Task  Force 
Study  Report  and  other  system  control  documents.” 
Phase  E  System  Testing  is  of  major  interest  because 
it  is  concerned  with  the  observation  and  assessment 
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of  complete  system  functions.  The  goal  of  this  system 
test  plan  was  to  bring  the  system  into  being  as  speci¬ 
fied,  to  operate  and  improve  it  under  test  conditions, 
and  to  provide  an  observational  basis  for  developer/ 
user  judgments  regarding  the  acceptability  of  demon¬ 
strated  capabilities  for  performing  mission  tasks. 
Objectives  for  achieving  the  goal  were: 

•  Verification  of  the  physical  characteristics  and 
functional  capabilities  of  the  system  as  these 
were  defined  in  requirement  documents: 

•  Measurement  of  system  performance  of  those 
functions  which  were  susceptible  to  available 
measurement  techniques,  in  order  to  provide 
baseline  performance  data  at  turnover  for  the 
information  of  the  using  operating  commands; 

•  Identification  of  system  changes  which  were 
implementable  within  authorized  resources  and 
schedules  of  the  system  test  program,  and 
adequate  documentation  of  change  recom¬ 
mendations  beyond  the  scope  of  developer 
resources; 

•  Provision  of  familiarization  with  the  test  system 
and  experience  in  manning  the  test  system 
under  simulated  conditions  of  COC  operations 
for  command/operator  personnel; 

•  Development,  exercise,  and  improvement  of 
operating  procedures,  to  include  normal  opera¬ 
tions,  system  degradation,  recovery,  and  manual 
backup; 

•  Demonstration  of  system  operations  under 
variable  conditions  of  readiness  for  handling 
live  and  simulated  inputs  with  the  express 
purpose  of  briefing  personnel  and  agencies 
having  a  need  for  knowledge  of  system  config¬ 
uration  and  test  progress,  and 

•  Preparation  for  system  performance  demonstra¬ 
tion  including  a  description  of  the  “as  delivered” 
configuration. 

Detailed  test  plans,  objectives  and  schedules  as 
well  as  implementation  methods  and  procedures  were 
stated  in  Test  Order  documents.  These  in  turn  were 
reduced  to  manageable  Mission  Orders  which  es¬ 
tablished  specific  objectives  and  test  conditions  under 
which  implementation  occurred.  Nine  Test  Orders 
and  a  plan  for  testing  systems  interfacing  with  the 
NORAD  COC  were  prepared,  supplemented  by 
approximately  23  Mission  Orders. 

The  plan  for  System  Performance  Demonstration 
(SPERD)  also  appeared  as  an  Appendix  to  the  Cate¬ 
gory  II  Test  Plan.  Performance  demonstration  was 
to  be  accomplished  through  a  command  post  exercise 
patterned  after  “problems”  of  a  type  employed  previ¬ 
ously  in  Group  I  and  Group  II  systems.  Some  Cate¬ 
gory  II  test  controls  for  managing  system  tests  were 


to  be  withdrawn  so  that  the  system  was  “at  the  dis¬ 
posal”  of  the  user/operator  in  the  performance  of 
mission  tasks.  Essential  SPERD  objectives  were  to: 

•  Provide  NORAD  with  an  opportunity  to  operate 
and  observe  the  Basic  NORAD  COC  in  an 
environment  free  of  test  constraints  so  that  a 
determination  could  be  made  of  the  degree  to 
which  the  system  had  achieved  readiness  for 
air  defense  operations,  and 

•  Conclude  Category  II  testing  and  support  sys¬ 
tem  turnover  plans. 

Documentation 

The  developmental  program  was  supported  by  a 
continuous  interest  in  keeping  all  aspects  of  develop¬ 
mental  activity  suitably  documented.  This  held  for 
broad  planning  documents,  system  specifications,  test 
orders  and  subordinate  test  procedures,  test  design 
and  methodology,  data  collection  and  result  report¬ 
ing,  and  phase  reviews. 

Of  particular  concern  in  the  present  context  are 
those  documents  which  established  a  framework  for 
the  conduct  of  experimental  and  test  activities.  T  he 
initial  documents  were  primarily  concerned  with 
broad  guidelines  for  undertaking  studies  in  a  labora¬ 
tory-like  environment.  They  entailed  conceptual  esti¬ 
mates  of  what  the  system  would  consist  of  and  how 
it  would  operate.  Experimentation  was  visualized 
as  an  exploratory  and  evolving  process  which  con¬ 
tributed  significant  data  to  advanced  planning  and 
final  system  design. 

Second  order  documents  included  phase  experi¬ 
mentation  plans,  special  test  orders,  and  the  test 
plans  required  under  Air  Force  Regulation  80-14. 

The  bulk  of  documentation,  after  plans,  consisted 
of  individual  test  reports,  phase  summary  and  evalua¬ 
tion  reports,  design  analyses,  and  design  recommenda¬ 
tions  at  all  system  levels. 

Experiments,  tests,  demonstrations  and  exercises 

To  this  point  it  has  been  necessary  to  develop  a 
structure  for  the  review  of  experiments,  tests,  dem¬ 
onstrations  and  exercises  conducted  in  evolving  the 
system.  Consequently  attention  was  given  to  the 
developmental  setting  within  which  these  activities 
occurred;  some  of  the  physical  and  functional  char¬ 
acteristics  of  the  experimental  system;  several 
methodological  considerations,  and  the  plans  and 
objectives  developed  for  conducting  experiments, 
tests,  demonstrations  and  exercises. 

The  term  experimentation  has  not  been  used  in  the 
classical  sense  which  would  imply  discretely  manipul- 
able  variables  and  laboratory  conditions  of  control. 
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However,  it  does  pertain  to  the  exploratory  activities 
concerned  with  establishing  and  utilizing  an  experi¬ 
mental  tool  capable  of  performing  functions  and  being 
manipulated  so  that  observations  regarding  its  im¬ 
mediate  character,  and  predictions  and  recommenda¬ 
tions  concerning  its  subsequent  character,  were  pos¬ 
sible.  Consequently  experimentation  in  this  instance 
refers  to  the  activities  carried  out  during  the  Phase 
C  and  C-l  interval,  whereas  testing  refers  to  the  ar¬ 
ray  of  tasks  and  actions  scheduled  under  the  Category 
II  Test  Plan.  Indoctrination  and  preparatory  activities, 
typified  by  SOSP  and  SPERD,  as  well  as  certain  test 
orders  and  training  sessions,  fell  under  the  headings 
of  demonstration  and  exercise. 

Experimentation 

Eaeh  objective  specified  for  Phase  C  experimenta¬ 
tion  was  addressed  during  the  experimental  series. 
Of  188  experimental  change  recommendations  pro¬ 
cessed  and  approved  during  this  phase,  40  were 
adopted  for  Phase  F  design.  On  the  basis  of  Phase  C 
experimentation,  it  was  judged  that  425L  as  planned 
for  the  initial  operational  system  would  improve 
NORAD  COC  operation.  Of  those  items  on  which 
experience  had  been  gained  in  Group  II,  many  met 
NORAD  requirements,  some  needed  elimination  or 
modification,  and  some  were  not  clearly  assessable. 
Significant  system  findings  in  summary  were: 

•  The  display  capability  of  the  experimental 
system  was  a  measurable  improvement  over  the 
Group  I  operation.  Timely  distribution  of  data 
and  rapid  retrieval  of  information  by  operators 
was  facilitated. 

•  Operating  procedures  needed  improvement  in 
order  to  exploit  system  capabilities. 

•  The  potential  for  higher  system  capacity  (over 
Group  1)  was  clear  particularly  in  the  volume 
and  variety  of  information  that  could  be  main¬ 
tained  and  displayed  for  command  utilization. 
More  timely  and  accurate  data  could  be  placed 
at  the  disposal  of  the  command  organization. 

•  Response  time  in  system  handling  of  certain 
critical  events  was  not  equal  to  Group  I  capa¬ 
bility  but  data  processing  design  changes  were 
expected  to  improve  this  condition. 

•  Some  processing  and  correlation  aids  demon¬ 
strated  in  the  experimental  system  provided 
capabilities  not  achievable  in  the  manual  sys¬ 
tem. 

•  Positional  organization  and  allocation  of  opera¬ 
tor  responsibilities  were  in  need  of  clearer 
definition. 

•  Details  of  display  manipulation  and  data  base 
content  would  require  some  changes  as  system 
operator-task  relationships  were  completed. 


An  additional  result  of  Phase  C  activity  was  the 
command  post  mock-up  design,  fabrication  and  review 
which  concluded  with  NORAD  approval  of  the  de¬ 
sign  and  instructions  to  proceed  with  specifications  for 
Group  III  procurement  and  modification  of  the  Group 
II  command  post  for  experimentation  and  test. 

Of  the  188  experimental  change  recommendations 
occurring  during  Phase  C,  equipment  accounted  for 
30,  general  program  and  documentation  36,  functional 
area  program  and  documentation  99,  support  and  utili¬ 
ty  programs  7,  facility-command  post  3,  operator 
methods  and  procedures  6,  and  system-general  7. 

Objectives  for  Phase  C-l  experimentation  were  not 
fully  realized  because  test  activities  under  C-l  test 
plans  were  inter  rupted  by  redirection  of  the  Group  II 
effort.  Nevertheless  approximately  20  command  post 
test  starts  were  accomplished  before  the  phase  was 
aborted.  Of  all  test  starts  only  one,  a  preliminary 
trial,  was  supported  by  an  operable  large  wall  dis¬ 
play.  All  others  simulated  large  group  display  utiliza¬ 
tion  by  assigning  the  large  command  post  display  to 
one  of  the  command  dais  display  consoles.  Positional 
procedures  prepared  in  advance  were  submitted  to 
trial,  modified  as  a  consequence  of  test  experience, 
and  applied  in  updated  form  to  succeeding  test  runs. 

Simulation  of  the  operational  program  system  dur¬ 
ing  the  latter  part  of  Phase  C-l  tests  permitted  devel¬ 
opment  and  practice  of  procedures  for  each  command 
dais  position.  Test  observation  and  data  collection 
techniques  were  established,  tried  and  improved.  An 
important  step  in  development  of  demonstrable  inte¬ 
grated  system  performance  was  achieved  in  the  first 
“formal”  Group  II  command  post  exercise  (patterned 
after  certain  Group  I  techniques).  The  system,  which 
this  time  included  an  operating  large  group  display, 
was  staffed  entirely  with  military  personnel.  Test 
management  and  demonstrations  were  concluded  to 
be  procedurally  reliable  for  conducting  the  balance  of 
the  command  post  tests. 

Some  interim  findings  upon  conclusion  of  the  C-l 
Phase  were  that  the  large  group  display  was  not  yet 
reliably  accessible  for  test  operations;  operator  skills 
which  had  an  important  bearing  upon  system  perfor¬ 
mance  were  in  need  of  a  systematic  program  of  devel¬ 
opment  and  maintenance  and  total  system  reliabil¬ 
ity-programs,  computer,  display  equipment,  per¬ 
sonnel,  operating  procedures  — at  this  point  was 
sufficiently  low  to  hinder  conformance  to  test  and 
demonstration  schedules.  However,  these  were  all 
attributed  primarily  to  the  Group  II  development 
stage  and  problems  of  simulating  Phase  1  program 
capabilities. 

Demonstrations  and  exercises 

Requirements  for  demonstrations  and  exercises. 
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including  training  sessions,  recurred  throughout  the 
development  program  and  frequently  appeared  as  ex¬ 
plicit  objectives  and  formally  scheduled  events  in  ex¬ 
perimentation  and  test  plans.  Some  of  the  earliest  dem¬ 
onstrations  concluded  with  NORAD  concurrence 
on  command  port  layout  design  and  recommendations 
for  procurement. 

Command  Post  Demonstrations  were  developed  in 
order  to  exhibit  typical  system  features  and  capabili¬ 
ties  for  observers  interested  in  design  and  operational 
aspects  of  the  system.  Two  demonstrations  were 
developed  and  employed. 

1.  Command  Post  Demonstration  1  (CP  Demo  1) 
was  designed  as  a  15-minute  exercise  to  introduce  the 
primary  capabilities  of  the  425 L  display  consoles 
and  the  SOSP  representation  of  the  3-E  Computer 
Program  design.  Demonstration  design  included  pro¬ 
gram  control  of  all  console  switch  actions  and  a  15- 
minute  magnetic  tape  recording  of  system  inputs. 
These  tape  inputs  generated  a  sampling  of  displays 
critical  to  mission  performance.  Appropriate  simu¬ 
lated  switch  action  “inputs”  were  included  to  demon¬ 
strate  operator  display  manipulation  and  information 
retrieval.  The  standard  presentation  entailed  running 
CP  Demo  1  for  approximately  15  minutes  followed  by 
30  to  40  minutes  of  “live”  display  console  operation 
by  observers. 

2.  Command  Post  Demonstration  2  (CP  Demo  2) 
was  designed  to  supplement  CP  Demo  1  with  a  45- 
minute  tape  showing  a  time-compressed  sequence  of 
peace-time,  attack,  and  battle  situations.  Subsequent 
changes  reduced  the  tape  to  30  minutes,  with  fewer 
inputs,  and  incorporated  the  large  wall  display.  Inputs 
consisted  of  a  variety  of  events  typifying  the  situations 
depicted.  Simulated  switch  action  “inputs”  were  used 
to  demonstrate  a  number  of  complex  operator  actions 
including  control  of  the  large  wall  display.  The  routine 
presentation  of  CP  Demo  2  involved  running  the  CP 
Demo  1  tape  for  15  minutes  followed  by  25  to  30 
minutes  of  the  CP  Demo  2  tape.  However,  during  ini¬ 
tial  demonstrations  the  wall  display  was  not  available 
due  to  installation  difficulties. 

SOSP  testing  concerned  with  development  and 
documentation  of  operating  procedures  and  familiar¬ 
ization  of  NORAD  COC  personnel  with  system  per¬ 
formance  in  a  simulated  operational  environment,  was 
largely  a  demonstration  phase.  A  total  of  55  successful 
and  partly  successful  runs  were  accomplished.  Com¬ 
mand  post  positions  were  repeatedly  exercised  over  a 
wide  range  of  simulated  operational  situations.  CP 
Demos  I  and  2  were  employed  frequently  during  the 
SOSP  interval  and  new  demonstrations  were  de¬ 
veloped. 


SOSP  experience  suggested  that  demonstrations 
and  training  exercises  be  clearly  separated  from  tests 
and  that  test  participants  be  shielded  from  visiting 
observers  and  non-participating  “experts.”  The  impor¬ 
tance  of  demonstrations  for  satisfying  a  variety  of  de¬ 
velopmental  requirements  was  made  evident  and  re¬ 
peatedly  confirmed.  The  need  for  a  stable  operator 
complement  and  systematic  positional  and  team  train¬ 
ing  opportunities  in  support  of  system  testing  and 
operational  system  development  was  firmly  establish¬ 
ed  during  the  SOSP  phase. 

Objectives  in  the  2-E  test  series  under  the  Category 
II  Test  Plan  included  the  familiarization  of  Group  I 
COC  personnel  and  the  Group  III  training  organiza¬ 
tion  regarding  all  aspects  of  system  operations  in¬ 
cluding  simulation.  Several  system  runs  were  allocated 
primarily  for  purposes  of  position  training  and  opera¬ 
tional  procedures  development.  Recommendations 
were  made  that  procedures  be  designed,  documented 
and  a  review  of  test  data,  showed  an  apparent  shor¬ 
tage  of  formally  specified  and  documented  position- 
oriented  procedures  which  could  be  submitted  to  test 
and  employed  as  operator  training  guides.  The  lack 
of  documented  procedures  at  this  stage  of  system  test¬ 
ing  and  the  consequent  difficulty  in  developing  ope¬ 
rator  proficiency  capable  of  exploiting  system  po¬ 
tential  were  considered  serious  barriers  to  the 
bchievement  of  levels  of  system  operation  permitting 
valid  and  reliable  measures  of  operator  dependent  sys¬ 
tem  performance  (Lesiw,  1965a).  A  recommendation 
was  made  that  procedures  be  designed,  documented 
and  verified,  and  systematic  training  be  undertaken 
if  the  objectives  of  performance  measurement  and  im¬ 
provement  were  to  be  realized  in  Phase  E  System 
Testing. 

During  final  testing  of  3-E  in  Group  1 1  four  test  ex¬ 
ercises  were  explicitly  concerned  with  procedures  and 
skill  development  problems.  However,  the  combina¬ 
tion  of  all  test  runs  provided  over  600  man-hours  of 
positional  familiarization  with  an  average  of  23  hours 
per  position  spent  in  the  performance  of  command 
post  and  divisional  operator  tasks  under  a  variety  of 
simulated  tactical  and  strategic  attack  conditions.  Ac¬ 
cumulated  experience  during  Group  II  system  tests 
amounted  to  manning,  under  simulated  battle  and  task 
stress  conditions,  of  more  than  one  full  week  of  prime 
shift  operations  with  a  Battle  Staff  directing  the  com¬ 
mand  post  and  divisional  areas  under  the  NORAD 
recommended  organizational  concept  of  COC  man¬ 
agement.  Supplementary  familiarization  included  ap¬ 
proximately  9  hours  of  continuous  participation  in  a 
Group  II  command  post  exercise  patterned  after 
Group  1  exercises  involving  COC  and  subordinate 
NORAD  echelons. 
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Group  111  Phase  E  system  tests  resulted  in  1160 
man  hours  of  system  familiarization.  Records  of  posi¬ 
tion  encumbents  indicate  that  at  least  68  persons  re¬ 
ceived  operational  position  experience.  Personnel 
rosters  at  the  beginning  of  Category  III  system  oper¬ 
ations  revealed  that  about  85  per  cent  of  these  re¬ 
mained  to  man  positions  in  this  follow-on  period. 

System  Performance  Demonstration  (SPERD)  con¬ 
stituted  the  primary  device  for  accomplishing  system 
turnover  to  NORAD  upon  conclusion  of  Category  II 
tests.  Design  of  the  demonstration  which  entailed 
participation  of  lower  NORAD  echelons,  had  its  orgin 
in  large-scale  command  post  exercises  carried  out  in 
Group  I. 

SPERD  was  considered  to  have  achieved  its  objec¬ 
tives  particularly  in  providing  NORAD  with  an  op¬ 
portunity  to  operate  and  observe  the  basic  NORAD 
COC  in  an  environment  relatively  free  of  test  con¬ 
straints.  The  success  of  the  demonstration  was  borne 
out  additionally  by  a  CMC  Technical  Approval  Dem¬ 
onstration  (TAD)  Board  finding  that  the  system  was 
acceptable  and  essentially  ready  for  operational  use. 
SPERD  also  served  as  a  source  of  data  to  permit  veri¬ 
fication  of  eertain  equipment  functions  and  interface 
requirements  not  possible  in  earlier  testing  due  to 
late  equipment  installation  schedules. 

SPERD,  while  not  fundamentally  concerned  with 
familiarization,  did  serve  to  integrate  personnel  in  an 
environment  whieh  netted  over  350  man  hours  of  ex¬ 
perience  in  24  operator  positions  including  many 
incumbents  of  the  initial  operational  system. 

Category  II  system  testing 

System  testing  under  the  Category  II  Test  Plan 
covered  a  period  of  approximately  17  months.  Test 
objectives  stated  in  the  test  plan  governed  all  system 
testing.  A  3-increment  program  package  paced  the  test 
effort.  The  initial  increment  involved  neither  live  in¬ 
terface  with  console  operators  nor  the  full  capability 
for  processing  external  messages.  Consequently  tests 
were  confined  to  program  shakedown  and  demonstra¬ 
tion  techniques  developed  previously  for  simulating 
console  operator  actions.  Twenty-three  test  runs  were 
accomplished  before  the  second  increment,  2-E,  was 
available  for  tests. 

Thirty  tests  were  completed  using  the  2-P  program 
by  addressing  all  objectives  specified  in  the  system 
test  plan.  Of  the  30  tests,  16  were  concerned  with  the 
task  of  verifying  the  achievement  of  physical,  func¬ 
tional,  procedural  and  organizational  requirements 
specified  for  the  system.  Performance  measurement, 
particularly  with  regard  to  the  handling  and  disposi¬ 
tion  of  specialized  system  data,  received  considerable 
test  attention.  Based  upon  2-E  System  Tests,  37 
change  recommendations  affecting  2-E,  3-E  (Group 


II),  and  3-E  (Group  III)  were  documented  dealing 
with  procedures,  training,  and  manning:  communica¬ 
tions  and  equipment:  program;  and  miscellaneous 
items  concerning  test  implementation  and  develop¬ 
mental  needs.  Fifty  per  cent  of  the  recommendations 
had  to  do  with  procedures,  training  and  manning. 

Observation  of  positional  performance  across  the 
system  during  2-E  tests,  and  a  check  of  test  data, 
showed  an  apparent  shortage  of  formally  specified 
and  documented  position-oriented  procedures  which 
could  be  submitted  to  verification  tests  and  employed 
as  operator  orientation  and  training  guides.  The  lack 
of  documented  procedures  and  the  consequent  diffi¬ 
culty  in  developing  operator  proficiency  capable  of 
exploiting  system  potential,  were  considered  serious 
deterrents  to  the  achievement  of  system  operating  lev¬ 
els  which  would  permit  valid  and  reliable  measures 
of  operator  dependent  system  performance  (Lesiw, 
1965a). 

Assuming  operable  hardware  with  known  limits,  a 
determination  of  what  the  system  could  do  was  large¬ 
ly  a  function  of  the  procedures  employed  and  the  skill 
and  experience  of  operators.  Performance  measure¬ 
ment  during  the  2-E  test  series  was  accomplished  by 
sampling  divisional  area  and  command  post  tasks,  and 
applying  relatively  inexact  measurement  techniques 
to  the  performance  of  developmentally  primitive  pro¬ 
cedures  by  operators  comparatively  unskilled  in  sys¬ 
tem  exploitation.  Therefore,  the  obtained  data  were 
not  indicative  of  proficient  system  performance  (what 
the  system  is  potentially  capable  of  doing)  but  rather 
of  how  the  system  worked.  What  the  system  was  cap¬ 
able  of  doing  remained  to  be  determined  after  proce¬ 
dures  and  skills  were  fully  developed. 

The  final  program  increment,  3-E,  supported  system 
tests  in  Group  II  and  Group  III.  Thirteen  tests  con¬ 
cluded  the  test  activities  scheduled  for  Group  11.  All 
tests  contributed  to  the  achievement  of  five  major  ob¬ 
jectives  of  system  testing.  Three  of  16  capabilities 
verified  under  2-E  system  tests  were  reconfirmed 
while  7  additional  user-oriented  requirements  were 
verified  under  3-E  Group  1 1  tests. 

Of  39  recommendations  made  during  the  3-El  Group 
II  test  scries,  13  were  concerned  with  changes  to 
the  computer  programs,  4  with  modifications  or  ad¬ 
ditions  to  the  then  current  equipment  configuration, 
and  the  remaining  22  pertained  to  the  personnel  sub¬ 
system  (procedures,  manning,  and  training).  Recom¬ 
mendations  regarding  procedures  were  greatest  in 
number  for  the  command  post,  most  having  to  do 
with  display  generation,  some  with  responsibility  al¬ 
location,  and  a  few  with  interaction  between  command 
post  and  divisions. 

Results  of  the  2-E  and  3-E  Group  II  system  tests 
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provided  a  sound  basis  for  continuation  of  system  tests 
in  Group  111,  including  preparation  for  the  demonstra¬ 
tions  of  system  performance  which  terminated  Cate¬ 
gory  II  tests.  Simulation  techniques  and  library  re¬ 
sources  were  deemed  adequate  to  test  requirements 
specified  in  Group  111  test  plans,  while  those  capabil¬ 
ities  established  in  the  Group  II  system  were  consid¬ 
ered  ready  for  installation  and  operational  extension 
in  Group  III. 

Testing  in  Group  III  under  the  Category  II  Test 
Plan,  assumed  an  integrity  not  realized  in  Group  11. 
The  pressure  of  firm  schedules  and  an  expected  end 
product  gave  an  easily  understood  and  respected  pur¬ 
pose  to  test  plans  and  activities.  Rapport  among  test, 
operator,  simulation  and  support  personnel  exceeded 
levels  achieved  at  any  previous  phase  of  experimen¬ 
tation  or  test.  Results  of  Group  III,  3-E,  testing  were 
considered  in  a  context  of  overall  Category  II  System 
tests  and  were  interpreted  in  terms  of  the  broad  system 
objectives  specified  in  the  Category  II  Test  Plan. 

Equipment,  program  and  operator  aspects  of  system 
performance  were  examined  separately  and  in  combi¬ 
nations.  Conclusions  regarding  capabilities  were  de¬ 
veloped  by  engaging  concepts  of  system  capacity, 
availability,  response  time  and  compatibility. 

Functional  verification  of  equipment  ( Did  it  con¬ 
form  to  specified  requirements,  and  were  alterations 
necessary?)  was  an  integral  part  of  all  Category  II 
tests  and  led  to  design  discrepancy  reports  and  pro¬ 
posals  for  design  changes.  Maintainability  of  the  sys¬ 
tem  equipment  complex  was  established  under  several 
Test  Orders.  Equipment  availability  was  determined 
for  major  components  (MC)  during  a  test  run  of  9 
days.  Major  components  included  at  least  one  central 
data  processor,  one  magnetic  drum,  a  display  data 
controller,  one  input/output  data  controller  with  16 
operable  input  and  2  operable  output  lines,  and  a  con¬ 
figuration  control  switch  Availability,  expressed  as  a 
per  cent,  was  found  to  be  high  in  an  8-console  config¬ 
uration,  decreasing  with  additional  consoles  up  to  15. 
The  large  wall  display,  scheduled  for  access  22  hours 
daily  during  this  test  interval  showed  an  availability 
comparable  to  the  major  components. 

Program  performance  was  investigated  as  an  aspect 
of  system  preparation  for  tests.  Standardized  test  ve¬ 
hicles  were  developed  consisting  principally  of  simu¬ 
lation  tapes  containing  a  broad  sampling  of  specified 
inputs,  also  procedures  for  inserting  operator  actions 
and  recording  detected  discrepancies.  In  a  nine-month 
period  concluding  with  termination  of  Category  II 
tests,  193  program  discrepancies  were  identified  and 
documented.  The  high  rate  was  attributed  primarily 
to  insufficient  communication  among  programmers 
when  programs  were  reorganized  or  recompiled  and 


reassignment  of  personnel  within  programming 
groups.  However,  the  correction  rate  for  documented 
program  errors  was  high  enough  to  compensate  for  the 
volume  of  discrepancies  permitting  the  system  test 
effort  to  proceed  according  to  schedule.  Furthermore, 
only  8  per  cent  had  a  significant  effect  upon  system 
testing  in  that  they  seriously  degraded  scheduled  tests. 
Most  of  these  were  of  the  program  loop  or  halt  type. 
Nineteen  per  cent  of  the  errors  had  to  do  with  message 
input/output  processing,  44  per  cent  were  concerned 
with  display  generation,  and  37  per  cent  affected  in¬ 
ternal  processing. 

Equipment/program  performance,  measured  in 
terms  of  response  time  associated  with  the  interaction 
of  central  data  processor  and  operational  programs, 
was  determined  by  varying  conditions  of  input  load¬ 
ing.  Of  several  input  variables  affecting  response 
time,  two  were  selected  for  control:  message  input 
rate,  and  operator  action  rate.  Using  the  maximum 
capability  of  the  source  equipment  and  communication 
lines,  and  maximum  capability  of  the  simulation  gen¬ 
eration  program,  simulated  message  and  switch  ac¬ 
tion  inputs  were  distributed  as  uniformly  in  time  as 
conditions  permitted.  Under  these  extremes,  equip¬ 
ment/program  governed  responses  to  switch  actions 
and  message  inputs  were  well  within  specified  limits. 

The  extent  to  which  the  computing  capacity  of  the 
system  is  utilized  was  also  measured  under  several 
conditions  of  loading.  The  measure  employed  was  the 
percentage  of  total  operating  time  during  which  the 
system  is  “idle”  (time  in  which  no  inputs  were  being 
processed  or  waiting  to  be  processed).  The  data  re¬ 
ported  gave  idle  time  percentage  as  a  function  of  six 
pre-selected  combinations  of  simulated  message  input 
load  and  simulated  operator  action  load.  The  range  of 
idle  time  percentages  were  such  that  currently  con¬ 
ceived  maximum  demands  utilized  about  one-quarter 
of  the  systems  computing  capacity,  while  under  the 
most  extreme  demands  conceived,  three-quarters 
would  be  used. 

Equipment/program/operator  performance  actually 
involved  verification  of  basic  NORAD  COC  opera¬ 
tional  capabilities.  A  verification  checklist  based 
upon  a  guide  developed  during  Group  II  system  tests 
provided  a  standard  for  determining  compliance.  Re¬ 
sults  were  obtained  for  five  organizational  functions 
and  a  special  capability  associated  with  system  opera¬ 
tion  during  training.  Positive  capabilities  for  per¬ 
forming  all  operational  functions  specified  were  dem¬ 
onstrated  by  the  system.  At  the  detail  level,  a  total 
of  154  position  tasks  were  observed;  145  were  con¬ 
firmed,  2  partially  confirmed,  and  7  were  unconfirmed 
in  comparison  with  pre-specified  criteria  for  task 
performance.  Problems  noted  in  verification  of  the 


NORAD  CO C  Development  287 


command  post  function  were  of  a  procedural  nature 
with  the  principal  difficulties  having  to  do  with 
techniques  for  data  base  monitoring  and  information 
exchange  (Lesiw,  1 965b). 

The  apparent  influence  of  message  input  rate  and 
simulated  defense  environment  upon  speed  of  re¬ 
sponse  and  accuracy  was  investigated  for  selected 
aspects  of  performance.  The  categories  of  obtained 
measures  included  responses  to  Command  Post 
queries;  report  of  condition  changes  to  the  Command 
Post;  elapsed  time  between  key  message  inputs  and 
their  appearance  in  command  post  displays;  data  base 
update  rates,  particularly  error  message  correction; 
and,  function  maintenance  and  restoral  action  time. 
Specific  results  are  not  essential  to  the  present  dis¬ 
course,  however,  the  measurement  categories  may 
be  of  methodological  interest. 

System  design  specifications  were  based  in  part  up¬ 
on  expected  characteristics  of  data  inputs  to  the  cen¬ 
tral  data  processor  (rates,  degradation  levels,  etc,) 
during  crisis  situations,  as  well  as  upon  expected 
characteristics  of  remote  users  of  CDP  outputs.  An 
examination  of  some  of  these  manifest  characteristics 
in  conjunction  with  the  system  design  which  emerged, 
provided  insight  regarding  compatibility  between  the 
system  and  the  real  world.  During  the  course  of  test¬ 
ing,  two  classes  of  input  data  were  available  for  analy¬ 
sis  and  assumed  to  be  representative,  with  respect  to 
time  distribution,  of  real  crisis  situations  as  currently 
conceived.  The  classes  were  operator  console  switch 
inputs,  and  certain  message  inputs  via  the  input/output 
data  controller  (IODC).  The  switch  inputs  were  ob¬ 
tained  during  realistic  simulation  tests  and  the  mes¬ 
sage  inputs  during  exercises  which  utilized  live  data 
from  remote  NORAD  installations.  The  sample  in¬ 
puts  were  considered  to  be  typical  of  periods  expected 
to  exhibit  overload  conditions. 

It  was  concluded  earlier  that  the  system’s  computing 
capacity  was  more  than  adequate  to  deal  with  im¬ 
mediately  envisioned  real  world  input  data.  Analysis 
of  the  input  samples  just  cited  suggested  a  model  to 
adjust  such  design  parameters  as  computing  speed, 
program  scheduling,  and  input  buffer  allocation,  in 
the  event  processing  requirements  increase  signif¬ 
icantly  in  the  future.  Operator  console  inputs  were 
observed  to  have  error  rates  (inputs  unacceptable  to 
the  computer  program)  which  had  no  noticeable  effect 
on  overall  system  performance  hence  no  incompati¬ 
bility  was  believed  to  exist  at  the  console/operator 
interface.  Message  error  detection  and  handling  (those 
rejected  by  the  IODC  and  those  accepted  by  the 
IODC  but  rejected  by  the  computer  program)  in¬ 
volved  manual  operations  in  the  correction  process. 
The  probability  of  IODC  rejects  forming  queues  was 


extremely  low  and  corrections  were  accomplished 
quickly.  Computer  detected  message  errors,  on  the 
other  hand,  were  more  likely  to  queue  and  took  longer 
to  correct.  Appropriate  recommendations  for  re¬ 
solving  this  incompatibility  with  real  world  demands 
were  made.  Computer-generated  message  outputs 
could  be  given  only  limited  study  hence  results  re¬ 
garding  compatibility  were  inconclusive.  Although 
compatible  interactions  were  established  among  cer¬ 
tain  message  outputs,  transmission  circuits,  and 
terminal  equipment,  problems  were  identified  and 
submitted  to  immediate  study  for  improvement  recom¬ 
mendations  to  be  implemented  during  post  Cate¬ 
gory  1 1  operations. 

Table  I  summarizes  the  Category  II  test  effort  by 
relating  major  test  objectives  to  the  numbered 
Test  Orders.  It  also  shows  whether  objectives  were 
achieved  as  a  direct  or  indirect  result  of  test  plans. 
Recommendations  for  improvements  as  a  conse¬ 
quence  of  the  Category  1 1  Test  Program  were  docu¬ 
mented  separately.  A  total  of  92  improvement  recom¬ 
mendations  were  made,  of  which  62  were  proposals 
for  specific  system  changes.  Thirty  took  the  form  of 
study  proposals.  Of  the  original  92,  45  were  concerned 
with  computer  program  modifications,  45  with 
changes  to  internal  system  equipment,  and  2  with  ex¬ 
ternal  interfacing  systems.  Of  the  62  proposals  for 
specific  changes,  40  were  funded  and  scheduled  as 
Category  II  testing  closed.  Thirty-six  of  the  62  dealt 
with  equipment  and  42  with  programs.  Display  con¬ 
soles,  the  large  wall  display,  and  display  generation 
programs  tallied  the  largest  numbers  of  change  pro¬ 
posals.  Of  the  30  study  proposals,  19  were  concerned 
with  computer  program  improvements,  9  with  equip¬ 
ment  improvements,  and  2  with  improvements  to 
external  interfacing  systems. 

Developmental  program  implications 

With  this  survey  of  the  developmental  effort  in¬ 
volving  experiments,  tests,  demonstrations  and  ex¬ 
ercises  essentially  complete,  it  should  be  possible 
to  review  the  nature  of  these  accomplishments  from 
a  utilitarian  point  of  view.  However,  it  must  be  clearly 
recognized  that  the  results  of  the  developmental  pro¬ 
gram,  particularly  their  interpretation,  arc  heavily 
dependent  upon  the  methodology  employed.  For  it 
was  the  rationale  contained  in  the  experimental  and 
test  plans  which  gave  purpose,  direction  and  systema- 
ticity  to  the  developmental  activities.  And  it  was  the 
techniques  and  procedures  developed  for  simulation, 
data  recording  and  documentation  which  laid  the 
foundation  for  conducting  developmental  system  oper¬ 
ations  and  collecting  relevant  system  data.  The  adapt¬ 
ability  of  the  methodology  injected  a  note  of  con¬ 
sistency  throughout  system  evolution  and  made  it  pos- 
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sible  to  adjust  the  developmental  effort  to  major 
changes  in  system  concepts  and  program  management. 

Experimentation 

The  preparatory  and  exploratory  activities  of  Phases 
C  and  C-I  established  an  experimental  system  which 
could  be  operated,  observed  and  described.  It  prompt¬ 
ed  development  of  simulation  techniques  which  could 
place  realistic  demands  upon  the  system,  including 
its  operators.  The  experimental  system  functioned  as 
a  diagnostic  instrument  in  that  it  exhibited  defi- 
ciences  requiring  immediate  corrective  action  along 
several  system  dimensions  — programs,  operator 
procedures,  equipments,  and  facilities  — before  devel¬ 
opment  could  continue  suitably.  Changes  were  in¬ 
stituted  and  evaluated.  Command  post  mock  up  and 
layout  design  analysis,  including  studies  of  periph¬ 
eral  command  post  displays,  were  accomplished  dur¬ 
ing  this  experimental  interval  which  resulted  in  prep¬ 
aration  of  specifications  for  the  Group  Ill  command 
post  installation.  Experimentation  with  various  con¬ 
figurations  of  the  command  post,  and  its  interaction 
with  other  elements  of  the  Combat  Operations  Center 


permitted  development  of  command  post  operating 
modes  and  concepts  of  integrated  system  operations 
necessary  to  follow-on  phases  of  effort.  The  general 
outcome  of  the  experimental  phase  was  a  test  system 
with  established  rules  for  conducting  system  opera¬ 
tions  manned  by  operators  indoctrinated  in  the  philos¬ 
ophy  of  the  test  program.  Included  was  the  technol¬ 
ogy  for  accomplishing  a  wide  assortment  of  test 
objectives  as  well  as  channels  for  introducing  and  im¬ 
plementing  system  change  recommendations. 

Demonstrations  and  exercises 

Planned  and  extemporaneous  demonstrations  were 
employed  throughout  the  developmental  program  in 
response  to  a  variety  of  needs.  Earliest  demonstra¬ 
tions  were  concerned  with  illustrating  system  design 
characteristics  and  capability  performance  particular¬ 
ly  for  those  requiring  design  or  functional  knowledge 
of  the  system  for  R&D  management  or  operational 
planning  purposes.  Demonstrations  were  also  em¬ 
ployed  as  indoctrination  tools  for  personnel  assigned 
to  man  the  system  as  well  as  for  technical  and  manage¬ 
ment  representatives  of  various  agencies  concerned 


Tabic  I.  Category  IT  Test  Objectives — Test  Order  Correlation 


Objectives 

1.  System  Preparation:  Program  Acceptance 

2.  System  Performance  Measurement 

a.  Operator  Independent 

b.  Operator  Dependent 

3.  Equipment  Functional  Verification 

4.  System  Operational  Capabilities  Verification 

5.  User-Operator  Personnel  Familiarization 

6.  System  Performance  Demonstration 

a.  Preparation 

b.  Accomplishment 

7.  System  Orientation  Demonstration 

8.  Procedures  Development 

9.  System  Improvement  Identification 


Test  Orderst 


303  307  308  31  1  313  314  315  316 


** 


** 


*  * 

* 

*  &  * 


** 


** 

** 

** 

* 

* 

* 

*  * 

** 

* 

** 

*  * 

* 

* 

* 

* 

** 

* 

** 

** 

* 

* 

** 

** 

* 

* 

NOTE:  *  Indirect  Objective  **  Direct  Objective 

"Test  Orders  310  and  312  were  considered  initial  increments  (2-E,  3-E  Group  II)  of  TO  314  for  this 
tabulation. 
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with  developmental  progress  of  the  425L  Program. 
The  most  notable  demonstration  from  a  utility  stand¬ 
point  was  the  highly  organized  and  extensively 
planned  System  Performance  Demonstration  that  con¬ 
cluded  Category  II  Testing  and  provided  a  vehicle  for 
assessing  the  readiness  of  the  Group  111  system  to 
assume  operational  tasks. 

Exercises  for  interim  estimates  of  system  effective¬ 
ness  were  developed  for  both  Group  II  and  Group 
III.  These  had  the  additional  objective  of  integrating 
personnel  functions  and  providing  opportunities  for 
organizational  skill  development.  Training  exercises 
for  both  operator  procedures  and  skill  development 
were  regularly  scheduled  features  of  the  test  phases. 
The  simulated  operational  support  phase  (SOSP)  was 
essentially  a  period  for  demonstrating  the  capacity 
of  the  otherwise  obsolete  Phase  C-l  system  to  re¬ 
semble  the  functional  configuration  proposed  by  the 
NORAD  Study  Group. 

It  is  appropriate  to  conclude  that  separate  demon¬ 
strations  and  exercises  satisfied  a  variety  of  require¬ 
ments  for  knowledge  of  the  system  without  interrupt¬ 
ing  or  imposing  upon  ongoing  system  experiments 
and  tests. 

Category  II  system  tests 

The  period  of  Category  II  tests  was  oriented  almost 
exclusively  toward  the  development  of  operational 
capabilities;  consequently  the  terminal  product  was 
a  system  ready  to  assist  in  the  performance  of  the 
NORAD  mission.  This  meant  that  the  test  program 
had  provided  confirmation  of  equipment  and  program 
design  specifications  and  recommendations  for 
changes  as  these  were  evident  from  test  experience. 
It  also  implied  that  performance  characteristics  had 
been  examined  and  that  performance  data  were  avail¬ 
able  regarding  system  response  time,  capacities  for 
handling  inputs  and  generating  outputs,  error  iden¬ 
tification  and  correction,  data  retrieval  and  display 
accuracy,  and  system  compatibility  with  the  real  en¬ 
vironment  within  which  it  had  to  perform. 

The  utility  of  this  phase  was  especially  manifest  in 
the  success  with  which  general  test  objectives  were 
addressed,  and  the  users  (NORAD)  readiness  to  as¬ 
sume  full  control  of  system  resources  and  capabilities 
at  the  conclusion  of  Category  II  tests.  In  addition 
the  system  recipient  was  provided  with  a  package  of 
simulation  and  exercise  technology  which  made  it 
possible  to  continue  simulated  operations  during 
Category  III  operational  testing  or  during  regularly 
scheduled  exercises  to  sustain  and  enhance  skills 
for  handling  the  system  routinely  and  on  a  crisis  con¬ 
ditioned  basis. 

Use  of  the  Group  II  test  facility  in  support  of 
experimentation,  demonstration  and  test  activities 


preceding  activation  of  Group  111,  contributed  signif¬ 
icantly  to  an  abbreviated  test  effort  and  early  opera¬ 
tional  employment  in  the  latter.  Category  II  testing 
in  Group  111  was  completed  in  less  than  five  months. 
A  program  of  product  improvement  consistent  with 
the  incremental  approach  was  submitted  for  follow-on 
implementation. 

Assessment  of  the  field  experimentation 

and  test  experience 

In  the  preceding  Section  on  Implications  it  is  evi¬ 
dent  that  the  results  of  the  developmental  program  of 
experiments  and  tests  were  viewed  in  a  positive  light. 
The  conclusion  drawn  is  that  the  program  was  a  suc¬ 
cessful  one  in  terms  of  both  process  and  product. 
However,  this  conclusion  does  not  imply  that  the  pro¬ 
gram  was  ideal  nor  that  methodological,  administra¬ 
tive,  and  management  problems  were  entirely  absent. 
There  were  indeed  many  problems  which  should  be  of 
interest  in  an  assessment  of  the  development  program 
experience. 

Experimental  system  control 

The  system  available  in  the  earliest  period  of  ex¬ 
perimentation  was  incomplete  and  functionally  un¬ 
reliable,  hence  it  could  not  be  considered  a  “good” 
representation  of  its  eventual  operational  counterpart. 
However  a  good  representation  at  this  early  date 
would  have  become  a  poor  one  in  the  light  of  sub¬ 
sequent  changes  introduced  by  the  NORAD  Study 
Group.  At  this  stage  of  experimentation  then: 

•  Additions  and  changes  were  continuous  in  al¬ 
most  all  system  dimensions —  hardware,  pro¬ 
grams,  data  handling,  personnel,  et  al.; 

•  Developmental  operating  characteristics  were 
unreliable,  making  performance  measurements 
tenuous,  especially  for  change  recommendations 
and 

•  Overt  design  and  procedural  problems,  par¬ 
ticularly  those  concerned  with  bringing  an 
operable  system  into  being,  were  resolvable 
and  constituted  the  prime  work  of  experimenta¬ 
tion. 

Simulation  technology,  fundamental  to  credible 
exercise  conditions,  evolved  with  the  system.  The 
experimental  phase  did  not  have  the  benefit  of  mili¬ 
tary  participation  (subject  matter  experts)  during 
initial  simulation  design.  The  result  was  lack  of  real¬ 
ism  in  problem  situations  which  in  turn  affected  oper¬ 
ator  motivation  and  hindered  procedure  develop¬ 
ment.  Extensive  user  involvement  in  simulation  de¬ 
velopment  at  a  later  date  was  disruptive  due  to  the 
magnitude  of  change  introduced  in  problem  design 
and  general  simulation  technique.  Nevertheless  the 
changes  were  constructive  and  improved  technology. 
It  can  be  said  that: 
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•  Sophistication  in  application  of  simulation 
techniques  was  partly  contingent  upon  specific 
experience  with  the  mechanics  of  utilizing  the 
system  under  development,  and  partly  de¬ 
pendent  upon  an  understanding  of  the  environ¬ 
ment  within  which  the  system  was  to  be  em¬ 
ployed.  No  less  important  was  an  awareness 
of  the  objectives  to  be  realized  as  a  result  of 
experimentation,  and 

•  Effectiveness  of  simulation,  both  in  content 
and  management,  increased  with  time.  Ef¬ 
ficiency  in  problem  design  and  production  also 
showed  a  corresponding  improvement. 

Operating  procedures  and  personnel  skills  could 
not  be  developed  systematically  under  field  experi¬ 
ment  conditions;  personnel  assignments  were  un¬ 
reliable  and  initial  procedures,  although  specified 
to  a  useful  level  of  detail,  were  not  effectively  ex¬ 
tended  and  documented,  hence  were  not  easily  veri¬ 
fied  or  passed  on  to  new  position  incumbents. 

Experimentation  as  a  phase  of  development  was 
not  fully  understood  or  accepted  as  an  important 
step  in  system  evolution;  consequently,  support  re¬ 
quirements  were  not  satisfied  as  readily  as  might  have 
been  expected.  At  the  same  time,  the  experimental 
program  did  not  enjoy  a  high  priority  in  budgetary 
and  developmental  schedule  considerations. 

Achievement  of  experimental  and  test  objectives 

Despite  situational  factors,  a  partly  installed  sys¬ 
tem,  programming  problems,  and  an  abruptly  can¬ 
celled  schedule  of  experiments,  the  objectives  of 
experimentation  were  realized  in  limited  form  and 
were  palpable  as: 

•  A  developmental  concept  of  system  operations; 

•  An  established,  working  system,  i.e.,  an  initial 
increment  capable  of  supporting  analyses, 
tests,  demonstrations  and  training  exercises; 

•  A  concept  of  simulation  for  the  system  and 
initial  development  of  simulation  technology; 

•  A  cadre  of  indoctrinated  experimental  system 
operators  not  fully  proficient  but  capable  of 
employing  developmental  procedures; 

•  Estimates  of  data  handling  capabilities  and 
selected  performance  characteristics; 

•  Program,  equipment  and  facility  changes,  and 

•  Resolutions  of  command  post  design  and  a  work¬ 
ing  basis  for  integration  of  command  post  func¬ 
tions  with  the  balance  of  the  system. 

The  objectives  of  Category  11  system  testing  were 
achieved  essentially  as  specified  but  required  the 
development  of  several  supporting  documents  which 
served  as  criterion  statements  and  performance 
measurement  guides.  Criterion  statements  extracted 


from  a  family  of  system  requirement  documents, 
were  necessary  to  accomplish  the  objective  dealing 
with  verification  of  physical  and  functional  capabili¬ 
ties.  A  performance  measurement  guide,  concerned 
with  measures  of  system  interaction  with  the  ex¬ 
ternal  environment  and  maintenance  of  internal  in¬ 
tegrity,  addressed  the  objective  of  “measurement  of 
system  performance  ...  in  order  to  provide  baseline 
performance  data . . . Both  documents  are  method¬ 
ological  curios  in  that  they  represent  explicit  en¬ 
deavors  to  deal  with  the  evaluation  issues  of  system 
performance  measurement  (What  to  measure?)  and 
criterion  development  (What  constitutes  performance 
adequacy?). 

Utility  of  the  Group  II  experimental  facility 

The  utility  of  the  experimental  facility  has  been 
substantiated  throughout  this  review.  Only  a  few 
additional  remarks  are  necessary  to  the  effect  that 
it: 

•  Provided  a  setting  for  early  experimental 
analysis  of  system  design  and  operating  char¬ 
acteristics; 

•  Supplied  an  austere  but  effective  test  bed  for 
initiating  Category  11  tests  sufficiently  in  ad¬ 
vance  to  influence  the  configuration  of  the 
Cheyenne  Mountain  Complex  NCOC  and  to 
simplify  transfer  of  integrated  Group  1  and  11 
operations  to  the  hard  site  shortly  after  oc¬ 
cupancy; 

•  Provided  some  initial  system  information  of  in¬ 
terest  to  the  NORAD  Cheyenne  Mountain 
Complex  Task  Force  in  its  deliberations  re¬ 
garding  the  direction  to  be  taken  in  NCOC  sys¬ 
tem  development; 

•  Facilitated  a  variety  of  design  changes  as  test¬ 
ing  progressed,  and  supported  evaluation  of  ap¬ 
proved  changes  upon  implementation; 

•  Served  as  a  relatively  economical  mock-up 
facility  for  command  dais  analysis,  design  and 
test,  as  well  as  a  model  for  exploring  variations 
in  facility  design  for  the  3-tiered  command  post 
in  advance  of  Group  111  construction,  and 

•  Provided  a  basis  for  establishing  initial  support 
requirements  for  operation  and  maintenance 
of  the  425L  segment  of  the  Group  111  NCOC. 

CONCLUSION 

From  the  experience  with  field  experimentation  and 
tests  associated  with  NORAD  COC  (System  425L) 
development  as  described  in  this  review,  it  is  judged 
that  the  contributions  of  these  developmental  activi¬ 
ties  facilitated  an  orderly  and  economical  evolution 
of  a  system  demonstrably  capable  of  performing  its 
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specified  mission.  Within  situational  and  physical 
constraints,  objectives  were  achieved  for  each  phase 
of  system  experimentation  and  test.  The  experimental 
facility  (Group  II)  did  in  fact  provide  a  relatively 
economical  and  effective  vehicle  for  establishing, 
exploring,  assessing  and  improving  initial  system 
concepts,  design  and  operations.  Experimental  data 
obtained  in  early  observations  of  system  behavior 
permitted  meaningful  forecasts  of  operational  system 
performance  and  estimates  of  personnel  subsystem 
requirements,  as  well  as  change  recommendations 
(hardware,  programs,  facilities)  which  were  incorpora¬ 
te  in  plans  and  specifications  for  the  system  sub¬ 
sequently  installed  in  the  Cheyenne  Mountain  Com¬ 
plex. 

It  is  also  true  that  the  program  of  experiments  and 
tests,  particularly  the  former,  was  hindered  by  de¬ 
velopmental  deficiencies  in  several  system  elements 
which  required  inordinate  amounts  of  preparatory 
and  shakedown  attention.  This  was  especially  evident 
in  early  programs  and  certain  equipments  such  as 
the  camera,  processor,  projector  system  which  gener¬ 
ated  large  screen  displays.  Redefinition  of  the  system 
and  redirection  of  the  development  program  as  a  re¬ 
sult  of  the  Cheyenne  Mountain  Complex  Task  Force 
Study  were  disruptive  influences  in  that  they  required 
a  new  cycle  of  planning,  a  revised  concept  of  system, 
and  a  re-orientation  of  developmental  activities  from 
experimentation  to  operational  development  testing. 
The  methodological  approach  employed  was  adequate 
to  bridge  the  shift  and  the  result  was  a  well-planned 
program  of  Category  1 1  testing  which  facilitated  transi¬ 
tion  from  Group  II  to  Group  III  and  rapid  turnover 
of  a  system  acceptable  to  the  recipient  user. 

Generalizations 

Based  upon  the  experiences  related  in  this  review, 
a  number  of  generalizations  appear  justified  regarding 
Held  experimentation  and  testing  of  large  scale  in¬ 
formation  systems  of  the  type  represented  by  the 
NORAD  Combat  Operations  Center.  Order  of 
presentation  docs  not  imply  priority. 

I.  It  is  advisable  to  establish  a  separate  facility 
representative  but  not  duplicative  of  the  physical 
and  functional  characteristics  of  the  subsequent  sys¬ 
tem  as  a  vehicle  for  experimentation  and  exploratory 
testing.  Preferably  this  experimental  facility  should 
be  established  early  enough  so  that  experimental  and 
test  findings  are  influential  in  setting  and  refining 
design  specifications  for  the  eventual  operational 
system  rather  that  in  correcting  defects  in  a  con¬ 
currently  installed  “operational"  system. 


2.  The  importance  of  documented  plans  for  experi¬ 
mentation  and  tests,  particularly  methodological 
considerations  — objectives,  experimental  design, 
data  collection  and  reduction,  and  result  report¬ 
ing— cannot  be  overstressed  since  these  constitute 
the  framework  for  management  and  implementation. 

They  also  may  provide  the  standards  for  assessment 
of  results  if  measurement  techniques  and  criteria 
for  evaluation  arc  integrated  with  objectives,  ex¬ 
perimental  design,  and  data  collection  and  recording 
procedures.  An  explicit  methodology  and  implementa¬ 
tion  plan  simplifies  the  task  of  assessing  effects  of 
major  changes  to  a  developmental  system  and  its 
associated  experimentation  and  test  program.  Short 
notice  reorganization  of  effort  is  also  facilitated. 

3.  The  development  of  simulation  technology  should 
be  accomplished  systematically  and  comprehensively 
as  early  as  possible  in  the  establishment  of  an  experi¬ 
mental  and  test  system  and  should  receive  studied 
attention  for  improvement  throughout  the  experi¬ 
mental  phase  not  only  because  it  directly  affects 
test  results  but  also  because  it  will  have  an  important 
role  in  the  post-test  system  as  a  device  for  system  ex¬ 
ercises  and  proficiency  evaluation. 

4.  In  conducting  experimental  system  runs  and 
scheduled  tests,  at  least  one  “system  engineer," 
intimately  acquainted  with  simulation  technology, 
program  characteristics,  system  equipment,  and  ob¬ 
jectives  and  procedures  for  the  scheduled  experiment 
or  test,  should  be  stationed  at  the  central  data  pro¬ 
cessor  to  manage  the  test  system.  The  role  of  this  test 
engineer  is  that  of  an  on-the-spot  diagnostician  of 
probable  system  problems,  familiar  with  routine  mal¬ 
functions  and  acceptable  conditions  of  system  degra¬ 
dation,  who  can  troubleshoot  and  correct  or  make 
immediate  arrangements  for  corrective  action  as 
emergency  needs  arise.  This  test  controller  will  make 
judgments  to  “abort"  or  to  continue  in  the  light  of 
prevailing  conditions.  He  should  function  as  an  “opti¬ 
mizer"  of  experimental  system  utilization. 

5.  The  credibility  of  simulation  is  enhanced  by  the 
presence  of  a  simulation  supervisor  who  is  thoroughly 
versed  in  the  subject  matter  aspects  of  combat  opera¬ 
tions  center  functions,  the  characteristics  of  the 
simulation  problem  to  be  employed,  the  immediate 
objectives  of  the  experiment  or  test  scheduled,  and 
the  organization  and  management  of  the  simulation 
center. 

6.  Complex  programs  designed  for  large  scale  in¬ 
formation  systems  of  the  type  encountered  in  the 
NCOC  development  effort  may  be  easier  to  handle 
in  the  experimentation  and  test  phase  if  they  are 
designed  and  produced  in  logical  or  functional 
increments  and  integrated  with  the  balance  of  the 
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experimental  system  in  successive  test  phases. 

7.  The  phase  approach  to  system  evolution  appears 
to  facilitate  organization  and  management  of  ex¬ 
perimentation  and  tests  because  it  promotes  periodic 
assessments  of  system  status  and  logical  junctures  for 
introducing  additions  and  changes.  The  assumption 
is  made  that  phase  terminations  are  complemented 
by  documented  statements  of  achievements,  analyt¬ 
ical  findings,  and  recommendations  for  follow-on 
increments  as  well  as  for  configuration  of  the  even¬ 
tual  operational  system. 
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Introductory  Remarks 

Ruth  M.  Davis 
Department  of  Defense 
Washington,  D.  C. 

Yes,  Intelligence  information  systems  do  have  much 
in  common  with  other  information  systems.  Most  of 
those  who  have  observed  Intelligence  information 
systems  recently  either  by  desire  or  by  direction  have 
made  it  their  objective  to  stress  the  commonality 
between  Intelligence  and  all  other  information  sys¬ 
tems.  Such  observations  are  akin  to  those  which  treat 
similarities  among  men  of  different  origins  and  dis¬ 
count  the  differences.  In  both  cases,  it  is  the  differ¬ 
ences,  which  when  neglected,  create  the  problems 
and  the  frictions.  One  feels  some  sympathy  towards 
the  professional  societies  of  England  who  finally 
were  led  several  centuries  ago  to  pay  mathematicians 
for  not  submitting  papers  containing  proofs  of  how  to 
trisect  the  angle.  So  it  is  that  one  looks  now  with 
delight  to  those  authors  who  have  the  insight  to  under¬ 
stand  and  present  the  important  differences  which  set 
apart  Intelligence  information  systems  and  their 
customers  — the  Intelligence  analysts. 

The  papers  of  Dr.  Hellner  and  Dr.  Wooster  pre¬ 
pared  for  the  Session  on  Information  Systems  Within 
the  Intelligence  Community  are  remarkable  in  the 
way  in  which  distinctive  features  of  Intelligence  and 
of  the  Intelligence  analyst  are  isolated  and  put  into 
perspective.  As  Dr.  Hellner  points  out,  “the  heart  of 
the  problem  of  Intelligence  analysis  is  to  draw  valid 
inferences  from  circumstantial  evidence."  The  In¬ 
telligence  analyst  must  make  use  of  “hypotheses  that 
invite  investigation"  in  his  reasoning  and  his  gen¬ 
eration  of  intelligence  products.  Intelligence  infor¬ 
mation  systems  must  assist  the  analyst  as  he  for¬ 
mulates  multiple  hypotheses  and  they  must  be  willing, 


contributing  partners.  Information  systems  which 
merely  store  and  retrieve  facts  will  be  “of  assistance 
to  the  annalist  but  not  to  the  analyst." 

Dr.  Wooster  compares,  in  his  always-effcctivc 
manner,  the  intelligence  analyst  to  the  miner  whose 
job  is  to  take  low-grade  ore  and  enrich  it  (“bene- 
ficiation  of  fairly  low  grade  ore")  There  is  the  impli¬ 
cation  throughout  his  paper  that  any  information  sys¬ 
tem  which  does  not  improve  this  ore  refinement-like 
task  of  the  Intelligence  analyst  should  be  itself  rele¬ 
gated  to  the  salt  mines.  He  also  states  quite  unequiv¬ 
ocally  that  mechanization  in  any  of  its  standard  forms 
“can  provide  a  partial,  but  only  a  partial,  amelio¬ 
ration"  of  the  troubles  besetting  the  Intelligence 
analyst. 

The  long  and  productive  experiences  that  Dr. 
Hellner  and  Dr.  Wooster  have  enjoyed  in  the  Intel¬ 
ligence  field  and  the  Information  Science  field  re¬ 
spectively  place  considerable  weight  upon  their 
observations. 

Perhaps  their  very  apt  reasoning  can  be  extended 
by  recalling  that  information  can  deal  only  with  four 
sets  of  phenomena.  Two  of  these  sets  relate  to  actual 
events  —  those  which  stand  for  themselves  — and 
which  are  studied  by  means  of  information  on  the 
functioning  or  behavior  of  the  organization,  the 
device,  the  object  itself,  and  by  means  of  information 
on  the  environment  in  which  the  functioning  or  be¬ 
havior  occurs.  The  other  two  phenomena  relate  to 
symbolic  events ;  namely,  those  that  stand  for  or  are 
seen  in  place  of  another  event.  In  this  ease,  obser¬ 
vations  must  be  made  on  communications  or  what  we 
may  call  the  symbolic  functioning  of  organizations, 
devices,  objects,  etc.,  and  on  the  environment  or  the 
context  in  which  the  communication  occurs. 

Most  military  information  systems  with  the  ex¬ 
ception  of  Intelligence  systems  handle  information 
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dealing  with  actual  events.  Logistics  information 
systems  manipulate  data  on  loading  of  cargo  vehicles 
on  inventories  of  items,  on  schedules  for  cargo 
delivery  and  the  like.  Planning  is  done  against  a  known 
environment  of  cargo  vehicles,  inventories,  ports, 
airports,  requirements  and  schedules. 

Command-control  information  systems  are  stocked 
with  data  on  readiness  status  of  military  units  and 
equipment,  on  locations  of  known  ships  and  aircraft, 
on  known  characteristics  of  artillery,  submarines, 
missiles,  etc.  Planning  is  done  against  a  known  back¬ 
ground  of  trajectories,  routes,  launch  times,  targets 
and  weapons.  Banking  information  systems  carry  data 
on  existing  status  of  checking  accounts,  savings  ac¬ 
counts  and  loans  and  on  completed  withdrawals  or 
deposits.  Planning  is  done  against  a  known  environ¬ 
ment  of  interest  rates,  congressional  legislation, 
stock  market  trends  and  the  like. 

In  all  these  just  cited  cases  the  actual  event  (or 
“thing'’)  can  be  checked  by  direct  observation  and 
the  environment  itself  is  subject  to  direct  observation. 

In  the  world  which  is  the  subject  of  Intelligence, 
the  actual  production  of  a  factory  in  a  foreign  un¬ 
friendly  country  cannot  be  directly  observed.  The 
color,  density  and  volume  of  smoke  pouring  from 
smokestacks  of  the  factory  must  be  viewed  from  a 
photograph  (a  symbol  of  the  factory  communicated 
via  photographic  means)  and  its  production  estimated. 
If  a  foreign  newspaper  article  can  be  located  which 
describes  a  new  university  laboratory  concerned  with 
welding  techniques  for  a  given  metal  which  is  located 
near  the  site  of  the  factory,  then  one  might  infer  the 
factory  output  from  circumstantial  evidence  In 
reality,  the  actual  event  of  interest  is  the  number  of 
operational  missiles  in  the  country  in  question,  the 
factory  output  is  the  symbolic  event  and  the  smoke 
visible  in  the  photograph  is  the  communication.  The 
Intelligence  analyst  must  somehow,  as  Dr.  Hellner 
states,  employ  categorization  and  classification 
techniques  and  draw  proper  inferences  through  in¬ 
terpretation  of  indicators  exhibited.  In  so  responding 
to  information,  the  Intelligence  analyst  cannot  use 
libraries  and  information  in  the  customary  way  and 
should  develop  some  specialized  tools  and  practices, 
the  lack  of  which  Dr.  Wooster  deplores. 

How  might  one  present  this  world  of  symbolic 
events  with  which  the  Intelligence  Community  deals 
daily?  Consider  perhaps  a  simplified  model  of  such 
a  world.  At  any  given  moment  in  time,  events  are 
occurring  which  are  either  isolated  or  members  of  a 


sequence.  As  they  are  happening,  information  is  being 
received  about  events.  This  information,  however, 
may  not  have  any  direct  bearing  upon  those  partic¬ 
ular  events  which  are  occurring  about  which  one 
desires  information.  The  third  element  in  this  model 
then,  is  that  set  of  events  deemed  of  importance.  Thus, 
the  triad  of  sets  of  events  which  forms  the  basis  of 
any  model  of  the  world  as  seen  by  the  Intelligence 
Community  is: 

1.  The  set  of  events  which  is  occurring  (symbolic 
events). 

2.  The  set  of  events  about  which  information  is 
being  received  (communication  of  symbolic 
or  actual  events). 

3.  T  he  set  of  events  about  which  knowledge  is  re¬ 
quired  (actual  events). 

The  implication  that  these  sets  of  events  may  be  com¬ 
pletely  or  partially  discrete  is  of  extreme  importance 
to  the  design  of  effective  information  systems  for 
Intelligence  analysts.  To  bridge  the  gap  between  the 
set  of  events  which  is  occurring  and  the  set  about 
which  information  is  received  requires  the  application 
of  inferential  logic  of  a  type  not  generally  under¬ 
stood.  No  information  system  has  yet  been  designed 
to  provide  assistance  to  the  Intelligence  analyst  in 
this  formidable  task. 

However,  Dr.  Blum,  in  his  very  lucid  paper  is  quite 
insistent  that  “computer  simulation  provides  a  power¬ 
ful  methodology”  for  performing  experiments  rapidly 
on  complex  information  systems.  He  rightly  points 
out  that  in  the  future  a  greater  emphasis  will  be  placed 
on  including  software  control  features  in  such  sim¬ 
ulation  experiments  and  certainly  in  any  information 
system  aiming  to  aid  the  Intelligence  analyst  soft¬ 
ware  will  predominate  over  hardware  in  importance. 
Dr.  Blum  also  states  that  it  is  advisable  to  utilize  the 
analytic  approach  with  its  mathematical  models 
whenever  possible  and  hence  lends  support  to  the 
sentiments  of  Dr.  Hellner  concerning  the  need  for 
experimenting  with  analytic  system  models  of  infer¬ 
ential  logic  to  support  the  Intelligence  analyst. 

The  three  papers  for  the  Session  when  viewed  as 
a  composite,  provide  an  intriguing  picture  and  an  ac¬ 
curate  picture  of  the  symbolic  world  with  which  the 
Intelligence  Community  must  deal,  of  the  special 
characteristics  which  make  for  a  good  Intelligence 
analyst  in  his  role  as  observer  of  this  symbolic  world 
and  of  a  powerful  tool  called  simulation  to  aid  in 
the  design  of  those  special  and  different  information 
systems  needed  to  aid  the  Intelligence  analyst. 


Evidence  and  inference  in  foreign  intelligence 


by  Maurice  H.  Hellner 
Defense  Intelligence  Agency 
Washington,  D.  C. 


INTRODUCTION 

As  President  Roosevelt  prepared  for  the  Yalta  Confer¬ 
ence  in  early  1945,  the  Joint  Chiefs  of  Staff  brought 
pressure  upon  him  to  persuade  Stalin  to  bring  Russia 
into  the  war  against  Japan  (Congress,  1951).  Their  in¬ 
sistence  was  based  primarily  on  eoneern  over  the  700,- 
000-man  Japanese  Kwantung  Army  in  Manchuria.  This 
army  had  the  reputation  of  being  an  extremely  efficient, 
well  commanded,  and  effective  fighting  force.  The  Joint 
Chiefs  feared  that  in  a  conquest  of  this  army  without 
Russian  help  U.S.  casualties  would  run  extremely  high. 
Roosevelt  followed  their  adviee  and  “persuaded”  the 
Russians  to  enter  the  war  against  Japan  by  agreeing  to 
Russian  territorial  acquisitions  in  the  Far  East  (Toland, 
1966). 

When  the  Russians  entered  the  war  against  Japan  in 
the  summer  of  1945,  the  highly-touted  Kwantung  Army 
crumbled  away  before  them.  After  the  Japanese  sur¬ 
render,  the  U.S.  learned  that  by  the  time  of  the  Yalta 
Conference  the  Kwantung  Army  was  depleted  and  by 
no  means  an  effective  fighting  force.  Most  of  its  elite 
units  had  been  transferred  piecemeal  to  the  battle  zones 
of  the  Pacific  as  the  need  for  them  arose.  In  January 
1943  for  example,  the  Japanese  had  14  divisions  in 
Manchuria.  Two  years  later,  10  divisions  plus  50,000 
men  in  smaller  units  had  been  shipped  out. 

This  is  a  discouraging  example  of  a  failure  of  intelli¬ 
gence  to  provide  required  information  to  high  military 
and  civilian  officials  at  critical  moments  in  history.  The 
failure  is  all  the  more  appalling  when  it  is  realized  that 
the  information  which  would  have  permitted  an  ae- 
eurate  estimate  of  the  situation  existed  in  bits  and  pieees 
within  the  U.S.  Intelligence  Community  but  was  never 
properly  collated.  As  U.S.  forees  captured  island  after 
island  in  the  Pacific  war,  the  Japanese  units  which  were 
destroyed  were  noted.  No  comprehensive  effort  was 
made,  however,  to  relate  these  units  to  units  whieh  were 
previously  known  to  have  existed  in  Japan  and  Man¬ 
churia.  Even  in  those  isolated  instances  where  this  was 
done  (as  in  the  ease  of  the  Japanese  50th  and  135th 
Infantry  Divisions  on  Saipan  when  Army  G-2  ob¬ 
served  that  these  were  units  of  the  Kwantung  Army), 


no  particular  significance  appears  to  have  been  at¬ 
tached  to  the  faet  (Farago,  1954).  Thus,  although  some 
of  our  highest  officials  opposed  our  efforts  to  bring  Rus¬ 
sia  into  the  Paeifie  war,  the  speetor  of  the  “powerful” 
Kwantung  Army  shaped  the  thinking  of  our  top  de¬ 
cision  makers. 

This  is  but  one  illustration  of  the  sobering  truth, 
learned  long  ago  in  the  physical  sciences,  that  in  the 
intelligence  field  the  mere  collection^  storage,  and  re¬ 
trieval  of  “faets”  is  no  guarantee  that  an  aeeurate  assess¬ 
ment  of  a  situation  will  follow  .  Except  in  rare  instances, 
the  faets  do  not  “speak  for  themselves.”  They  must  be 
correlated  in  such  a  manner  as  to  provide  the  basis  for 
valid  inferences.  The  heart  of  the  intelligence  problem 
is  to  develop  techniques  and  procedures  for  doing  just 
this.  Direet  observation,  and  sometimes  intuition,  are 
important  pathways  to  knowledge,  but  the  bulk  of  hu¬ 
man  knowledge  is  obtained  by  inference. 

The  foregoing  should  exercise  a  restraining  influence 
on  those  who  believe  that  modern  computer  technology 
can  provide  a  major  breakthrough  in  intelligence  proc¬ 
essing  capabilities  merely  by  increasing  one's  ability  to 
store  and  retrieve  data.  The  intelligence  processing 
problem  consists  of  considerably  more  than  providing 
expanded  library  services  for  an  increasing  volume  of 
information  to  a  large  number  of  analysts.  It  is  an 
utterly  superficial  view  that  truth  is  to  be  found  by 
merely  collecting,  storing  and  retrieving  faets. 

Although  there  is  considerable  literature  on  intelli¬ 
gence  operations  down  through  the  ages,  ineluding  a 
great  deal  on  methods  of  collection  and  observation, 
the  literature  of  the  trade  contains  virtually  nothing  on 
the  methodology  of  analyzing  the  data  that  has  been 
collected.  It  is  a  strange  paradox  that  in  the  literature 
and  folklore  of  intelligence  the  heroes  have  invariaoly 
been  the  collectors  of  information  (e.g.,  the  spies), 
whereas  in  the  literature  and  folklore  of  crime  the  heroes 
have  nearly  always  been  the  effective  analysts  of  infor¬ 
mation  (e.g.,  the  detectives).  Throughout  the  years  of 
recorded  history  there  are  startling  examples  of  ae¬ 
eurate  intelligence  estimates  and  predictions,  but  they 
are  nearly  always  based  on  the  acquisition  of  a  few'  items 
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of  dramatic  information  from  a  single  source  rather 
than  through  the  painstaking  correlation  and  analysis 
of  obscure  and  fragmentary  data  from  many  sources. 
The  U.S.  victory  at  the  Battle  of  Midway,  for  example, 
was  due  in  major  part  to  an  accurate  intelligence  pre¬ 
diction,  but  this  prediction  was  based  on  the  acquisition 
of  the  Japanese  battle  plan  rather  than  through  the 
piecing  together  of  fragmentary  data  (Fuchido,  1955). 

One  may  well  ask  at  this  point  whether  all  inferences 
in  intelligence  are  not  destined  to  remain  more  artistic 
than  scientific.  At  best,  intelligence  today  is  a  discipline 
in  the  process  of  formation.  The  major  obstacle  to 
progress  lies,  not  so  much  in  computer  technology 
(although  major  improvements  are  required  here), 
but  in  developing  methodology  for  the  analytic  process. 
Today,  systems  designers  attempting  to  develop  com¬ 
puter  systems  for  processing  intelligence  data  are  forced 
first  to  attempt  to  develop  the  procedures  of  analysis 
before  they  can  proceed  with  their  task.  It  is  as  though 
the  systems  designers  attempting  to  apply  computer 
technology  to  a  mathematical  problem  had  first  to  de¬ 
velop  the  formal  logic  of  mathematics  before  they  could 
proceed. 

Nature  of  the  problem 

It  has  been  pointed  out  by  scores  of  writers  that  the 
first  step  in  developing  any  methodology  is  to  define 
the  problem — to  isolate  the  questions  and  issues  in¬ 
volved.  We  have  already  suggested  that  the  mere  stor¬ 
age  and  retrieval  of  data  is  not  the  basic  problem  in 
intelligence  analysis.  The  basic  problem  is  to  develop  a 
methodology  which  will  facilitate  the  making  of  valid 
inferences  from  the  available  evidence. 

One  may  object  at  the  outset  that  this  is  true  of  any 
field  of  human  investigation,  and  so  it  is.  In  the  intelli¬ 
gence  field,  however,  the  process  of  drawing  valid 
inferences  from  available  data  is  a  particularly  complex 
one,  and  one  that  has  not  been  studied  in  depth.  The 
complexity  arises  from  two  principal  causes:  (1)  the 
nature  of  the  raw  information;  and  (2)  the  character 
of  the  processing  requirements.  These  may  be  sum¬ 
marized  as  follows: 

1 .  Nature  of  Raw  Intelligence  Data: 

(a)  Wide  variation  in  form,  content,  timeli¬ 
ness,  and  reliability. 

(b)  Excessive  amount  of  complicating  detail. 
Nearly  every  ‘‘indicator”  can  be  confronted 
by  other  “indicators”  which  contradict  it. 

(c)  Extremely  high  volume. 

(d)  Large  gaps  and  inconsistencies. 

(e)  Deliberate  deception  and  disguise  of  vital 
elements. 

(f)  Non-controllable  information  sources. 

2.  Character  of  Processing  Requirements: 


(a)  Great  importance  of  random  and  rare  events. 

(b)  Unpredictability  of  prccossing  load. 

(c)  Tendency  toward  rapid  changes  in  focus  of 
attention. 

(d)  Wide  variety  of  user  requirements. 

(e)  Severe  time  restrictions  on  many  processing 
tasks. 

Superimposed  on  all  of  these  factors  are  a  number  of 
complications,  many  of  which  are  unique  to  the  field 
of  intelligence  processing: 

1.  Physical  and  social  scientists  search  for  indicators 
of  classes  of  events.  Intelligence  analysts  are  more  con¬ 
cerned  with  indicators  of  specific  events.  This  raises  an 
acute  methodological  problem.  How  can  we  determine 
the  probability  of  a  single  case  (a  coup  d’etat,  a  sur¬ 
prise  attack,  etc.)?  To  put  it  another  way,  how  can 
the  scientific  method  which  has  been  developed  to 
analyze  the  universe  of  recurrent  events  be  adapted  to 
permit  valid  predictive  inferences  pertaining  to  political 
and  military  events  which  are  unique  and  nonrecurrent? 

2.  Although  each  political  and  military  event  is 
unique,  it  is  a  mental  and  physical  impossibility  to  deal 
with  each  event  as  completely  unique.  Classification 
and  categorization  are  fundamental  to  human  under¬ 
standing.  The  entire  concept  of  “normal”  and  “ab¬ 
normal”  situations  implies  categorization. 

3.  “Categorization”  in  the  intelligence  field  is  an 
extremely  delicate  process.  The  rules  for  defining  in¬ 
dividual  categories  are  influenced  markedly  by  the  es¬ 
timate  of  the  situation  the  analyst  already  has  in  his 
mind.  The  history  of  intelligence  processing  reveals 
quite  clearly  that  important  pieces  of  information  are 
often  subsumed  in  the  wrong  category  for  no  other 
reason  than  that  the  prevailing  estimate  of  the  situation 
made  it  logical  to  do  so.  This  is  perhaps  the  single 
greatest  weakness  of  intelligence  processing  in  totali¬ 
tarian  countries  where  the  “facts”  which  are  collected 
are  often  misinterpreted  because  the  official  party  line 
misguides  the  categorization  process.  Thus,  the  USSR 
was  surprised  by  the  German  invasion  of  1941,  in 
spite  of  ample  evidence  that  German  troops  were  mass¬ 
ing  on  the  frontier,  because  the  official  party  line 
emanating  from  Stalin  was  that  the  Germans  would  not 
attack  (Seth,  1964).  It  is  interesting  to  note,  however, 
that  even  British  intelligence  fell  into  the  same  trap  on 
that  occasion.  The  Joint  Intelligence  Committee  as  late 
as  one  month  before  the  Nazi  invasion  of  Russia,  cate¬ 
gorized  the  reports  of  large-scale  Nazi  troop  concentra¬ 
tions  on  the  Soviet  border  as  part  of  the  pressure  Hitler 
was  putting  on  Stalin  to  provide  additional  strategic 
materials  to  blockaded  Germany.  This  was  done  because 
the  offical  British  estimate  of  the  situation  was  that  there 
was  a  large  community  of  interest  between  Germany 
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and  Russia  in  overrunning  and  dividing  the  British 
Empire,  and  that  it  was  therefore  illogical  for  Germany 
to  attack  Russia  at  that  time  (Churchill,  1950). 

4.  An  individual  “indicator'1  from  which  an  inference 
may  be  drawn  has  only  a  probability  relationship  to 
any  given  situation.  No  one  indicator  is  likely  to  be 
conclusive.  Each  indicator,  by  itself,  is  susceptible  to  a 
number  of  interpretations.  When  one  attempts  to  for¬ 
mulate  a  profile  of  indicators,  it  is  difficult  to  deter¬ 
mine  the  confines  of  the  profile.  This  is  particularly 
true  because  implication  is  a  non-sym metrical  relation¬ 
ship.  The  reverse  of  “If  ‘p’  then  ‘q’  "  is  not  necessarily 
so. 

5.  The  main  stream  of  events  and  indicators  fre¬ 
quently  obscures  important  subordinate  events.  Thus, 
the  U.S.  Intelligence  Community  predicted  quite  ac¬ 
curately  the  time,  place,  and  direction  of  the  main 
Japanese  thrust  at  the  beginning  of  our  involvement  in 
World  War  II.  It  warned  our  decision  makers  that 
Japan  was  likely  to  go  to  war  either  the  last  week  in 
November  or  the  first  week  in  December  1941  and 
that  the  main  attack  would  be  directed  south  toward 
Indonesia,  the  Philippines,  and  Malaya  (Wohlstettcr, 
1962).  It  became  so  obsessed  in  its  analysis  of  all  “in¬ 
dicators"  of  this  attack,  however,  that  it  did  not  establish 
the  required  the  required  collection  and  processing 
posture  for  detecting  the  subordinate  attack  against 
Pearl  Harbor. 

6.  Accurate  analysis  of  raw  information  is  sometimes 
complicated  by  the  fact  that  two  or  more  major  events 
arc  developing  at  the  same  time.  In  such  cases,  the 
“categorization"  of  indicators  becomes  extremely  con¬ 
fusing  because  some  indicators  may  point  to  one  event, 
some  to  the  other,  some  to  both,  and  some  to  neither. 

7.  International  political  and  military  affairs  are  char¬ 
acterized  by  action  and  reaction.  It  is  generally  im¬ 
possible  to  analyze  accurately  the  likely  courses  of 
action  of  foreign  countries  except  in  relation  to  actions 
and  policies  of  other  countries.  Moreover,  the  “facts" 
our  estimators  look  at  are  almost  certainly  not  identical 
to  those  of  the  foreign  planners.  During  1941,  for 
example,  the  pattern  of  action  and  reaction  between 
the  Japanese  and  American  governments  became  pro¬ 
gressively  more  complex.  By  and  large,  U.S.  intelligence 
officers  in  Hawaii  were  seriously  handicapped  in  for¬ 
mulating  their  estimates  of  impending  Japanese  actions 
by  their  dearth  of  information  on  U.S.  diplomatic 
moves. 

8.  In  intelligence,  we  arc  dealing  primarily  with 
descriptive  data.  We  have  very  little  information  per¬ 
taining  to  casual  relationships.  This  considerably  com¬ 
plicates  the  problem  of  drawing  valid  inferences. 

9.  In  intelligence,  transient  relationships  and  charac¬ 
teristics  arc  often  more  important  than  permanent  or 


invariant  ones.  Thus,  the  transient  relationships  between 
a  group  of  ships  in  a  task  force  may  be  far  more 
meaningful  than  any  formal  order  of  battle.  There  is, 
however,  a  great  poverty  in  methodology  for  structuring 
data  bases  to  accommodate  transient  characteristics  and 
relationships. 

10.  Normally,  the  term  “evidence"  refers  to  an  ac¬ 
cumulation  of  data  and  the  term  “inference"  refers  to 
propositions  or  conclusions  which  were  not  included  in 
the  original  data  but  which  were  logically  deduced,  in¬ 
ferred,  or  extracted  from  it.  Such  a  distinction,  how¬ 
ever,  has  a  deceptive  clarity  in  most  professional  dis¬ 
ciplines,  and  especially  in  intelligence.  In  “real  life," 
yesterday's  inferences  are  today's  basic  data.  Inferences 
and  facts  become  integrated  in  such  a  manner  that  it 
is  impossible  to  separate  them  neatly.  More  significant, 
in  many  ways  subtle  and  obvious  past  inferences  shape 
current  “facts." 

11  In  the  real  world,  a  solid  prediction  often  goes 
sour  because  of  the  intervention  of  an  accidental  event 
— a  heart  attack  by  a  key  political  figure,  a  plane  crash 
which  upsets  plans  for  a  coup  d’etat,  a  flood,  etc.  Be¬ 
cause  of  this,  the  best  one  can  hope  for  in  intelligence 
is  prediction  in  terms  of  probability. 

12.  To  the  academic  scientist,  time  may  not  be  of 
particular  consequence.  He  is  after  a  high  degree  of 
confidence  in  his  results,  and  whatever  time  it  takes  to 
get  the  degree  of  confidence  he  is  after,  he  can  usually 
take.  The  intelligence  analyst,  on  the  other  hand,  must 
frequently  reach  his  best  possible  conclusion  in  a  lim¬ 
ited  time. 

13.  Finally,  there  is  the  age  old  problem  of  under¬ 
standing  behavior  that  is  alien  to  one's  own  culture. 
Any  attempt  to  develop  simulations,  models,  or  pro¬ 
files  of  indicators  must  face  the  fact  that  our  criteria 
and  belief  systems  may  be  inaccurate  and  unrealistic 
with  reference  to  a  particular  foreign  society.  Allen 
Dulles,  the  former  Director  of  Central  Intelligence, 
expressed  this  point  very  colorfully  when  he  said: 
“Actions  and  reactions  can  no  longer  be  estimated  on 
the  basis  of  what  we  ourselves  might  do  if  we  were  in 
Khrushchev’s  shoes,  because,  as  we  have  seen  at  the 
United  Nations,  he  takes  off  his  shoes  (Dulles,  1963). 

A  pproach 

One  may  well  ask,  in  view  of  this  long  catalog  of 
obstacles  and  difficulties,  whether  intelligence  pro¬ 
cessing  is  not  destined  to  remain  more  in  the  nature  of 
an  art  than  a  science.  Game  theory,  model  building, 
statistical  techniques,  content  analysis,  simulation,  and 
operations  research  all  appear  to  have  application  to 
the  field  of  intelligence,  but  only  in  restricted  areas. 
By  and  large,  it  docs  not  appear  possible  to  formulate 
any  “laws  of  events"  in  the  military  and  political  area 
in  quantitative  terms. 
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In  a  very  real  sense  the  heart  of  the  problem  of  in¬ 
telligence  analysis  is  to  draw  valid  inferences  from 
circumstantial  evidence.  If  approached  in  this  manner, 
it  is  believed  that  progress  can  be  made  in  developing 
a  methodology.  As  used  here,  the  term  “circumstantial 
evidence”  is  contrasted  with  what  may  be  described  as 
evidence  bearing  directly  on  the  main  problem.  It  de¬ 
notes  evidence  concerning  certain  facts  which,  so  to 
speak,  surround  the  main  event.  It  is  like  piecing 
together  the  pieces  of  a  picture  puzzle,  but  a  picture 
puzzle  of  which  many  parts  arc  missing,  while  others 
have  been  damaged,  distorted,  or  faked. 

It  is  true  that  in  the  vast  majority  of  cases  in  which 
reasoning  is  based  on  circumstantial  evidence  there  is 
no  possibility  of  generalizing  the  result.  That  is  so  be¬ 
cause  the  majority  of  the  cases  dealt  with  arc  such  as 
are  not  likely  to  recur  in  exactly  the  same  form.  This 
is  precisely  the  situation  we  find  in  intelligence. 

The  characteristic  feature  of  reasoning  from  circum¬ 
stantial  evidence  consists  of  the  construction  of  a 
system  for  which  the  various  items  of  evidence  form 
coherent  parts.  We  begin  with  a  suggested  explanation 
or  solution  of  the  problem.  Such  tentative  explanations 
arc  suggested  by  something  in  the  subject  matter  and/or 
by  our  previous  knowledge.  When  they  are  formulated 
as  propositions  we  normally  call  them  “hypotheses.” 
The  function  of  a  hypothesis  is  to  direct  our  search  for 
order  among  facts,  particularly  for  order  among  con¬ 
fusing  facts. 

It  is  perhaps  strange  that  although  intelligence 
analysts  have  borrowed  many  scientific  methods,  they 
have  shown  an  odd  reluctance  to  resort  to  the  formal 
use  of  hypotheses.  To  many,  a  hypothesis  has  seemed 
like  a  commitment,  a  pre-judgment,  when  the  mind 
should  be  kept  open  for  the  “facts.”  They  arc  afraid 
that  the  use  of  hypotheses  will  lead  them  into  the 
danger  of  forcing  the  facts  to  fit  the  theory.  They  feel 
that  any  given  situation  should  be  studied  without 
prejudice  or  presupposition. 

Yet,  scientists  learned  long  ago  that  nature  gives  no 
reply  to  a  general  inquiry.  She  must  be  interrogated  by 
questions.  The  observer  can  only  observe  that  which 
he  has  been  led  by  some  hypothesis  to  look  for.  Charles 
Darwin  expressed  this  concept  very  well:  “About  30 
years  ago  there  was  much  talk  that  geologists  ought  to 
observe  and  not  to  theorize;  and  I  well  remember 
some  one  saying  that  at  this  rate  a  man  might  as  well 
go  into  a  gravel  pit  and  count  the  pebbles  and  de¬ 
scribe  the  colors.  How  odd  it  is  that  anyone  should  not 
sec  that  all  observation  must  be  for  or  against  some 
view  if  it  is  to  be  of  any  service  (Darwin,  1903). 

It  is  frequently  stressed  that  the  intelligence  analyst 
must  be  open-minded,  but  open-mindedness  should  not 
be  construed  to  mean  mental  vacuity.  There  is  an  im- 


vestigation  and  fixed  theories  that  control  investigation. 
A  hypothesis  is  really  a  mental  attitude,  an  approach 
portant  difference  between  hypotheses  that  invite  in¬ 
to  the  categorization  of  facts.  It  is  a  method  of  as¬ 
suming  certain  situations  experimentally  in  order  to 
test  the  result.  The  hypothesis  may  very  well  prove  un¬ 
satisfactory  and  be  discarded. 

It  cannot  be  denied,  of  course,  that  there  is  a  subtle 
tendency  for  one  to  bestow  a  sort  of  parental  affection 
upon  an  intellectual  offspring.  If  this  tendency  is  not 
resolutely  offset,  the  usefulness  of  a  hypothesis  will 
vanish. 

One  means  of  control  is  the  use  of  multiple  hypoth¬ 
eses.  By  employing  this  method  the  analyst  becomes 
the  parent  of  a  family  of  hypotheses,  and  thereby  can¬ 
not  fasten  his  affections  unduly  upon  any  one.  The 
problem  with  this  approach  in  the  past  has  been  that 
the  human  mind  has  extreme  difficulty  in  pursuing  two 
or  more  lines  of  reasoning  at  the  same  time.  Simul¬ 
taneous  vision  from  different  standpoints  appears  to 
be  a  virtual  impossibility  for  the  unaided  human  mind. 

The  introduction  of  computer  technology  into  the 
processing  of  intelligence  information  may  well  provide 
the  assistance  to  the  human  mind  that  is  needed  for 
employing  the  method  of  multiple  hypotheses.  If  so, 
it  is  evident  that  this  will  require  a  man-machine  re¬ 
lationship  of  an  order  which  has  not  been  achieved  thus 
far. 

The  requirements  for  such  an  approach  arc  rather 
demanding.  Some  of  these  arc: 

1  An  on-line,  time-sharing  computer  system  that  of¬ 
fers  considerably  more  than  the  mere  retrieval  of  a 
specified  item  of  information  on  demand.  It  must  pos¬ 
sess  a  query  capability  which  permits  the  testing  of 
hypotheses. 

2  Data  bases  which  arc  structured  not  only  to  permit 
the  categorization  of  data  according  to  common  char¬ 
acteristics,  but,  more  important,  to  permit  the  dis¬ 
covery  of  connections  whereby  one  item  of  data  can 
be  related  to  another.  A  useful  data  base  in  intelligence 
will  not  be  achieved  by  categorization  alone. 

3  Data  bases  which  are  structured  not  only  to  ac¬ 
commodate  permanent  characteristics  and  relationships, 
but  also  transient  characteristics  and  relationships.  In 
a  fast-moving  world,  transient  characteristics  and  rela¬ 
tionships  arc  frequently  much  more  important  than  per¬ 
manent  ones. 

4  Extensive  use  of  “feedback”  so  that  valid  infer¬ 
ences  can  be  fed  back  into  the  data  base. 

5  Research  into  the  kinds  of  events  which  are 
reasonably  predictable  on  a  probability  basis.  A  begin¬ 
ning  could  be  made  by  ranking  important  kinds  of 
events  in  terms  of  their  probable  predictability. 
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CONCLUSION 

Computer  systems  which  offer  little  more  than  the 
mere  storage  and  retrieval  of  “facts”  arc  of  limited 
utility  in  the  processing  of  intelligence  information. 
They  can  provide  a  certain  amount  of  assistance  to  the 
annalist  but  not  to  the  analyst.  They  can  provide  very 
little  assistance  in  the  formulation  of  valid  inferences, 
particularly  predictive  inferences. 

Intelligence  today  is  a  discipline  in  the  process  of 
formation.  The  techniques  employed  arc  a  mixture  of 
that  which  is  unique  to  intelligence  and  that  which 
is  not.  The  mixture  is  certainly  unique.  Future  progress 
in  this  field  will  be  heavily  dependent  upon  the  de¬ 
velopment  of  methodology  and  technology  for  draw¬ 
ing  valid  inferences  from  raw  information  which  is 
generally  incomplete,  contradictory,  of  varying  ac¬ 
curacy,  and  often  faked. 
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INTRODUCTION 
The  need  for  simulation 

It  has  been  conceded  that  very  little  scientific  knowl¬ 
edge  exists  about  information  systems  although  much 
has  been  written.  The  literature  in  this  field  contains 
a  preponderance  of  philosophy  and  speculation  and 
the  overall  impression  is  that  progress  is  painfully 
slow.  What  progress  we  have  is  derived  mainly  from 
experience  in  the  development  of  actual  information 
systems.  Such  systems  characteristically  have  a  de¬ 
velopment  cycle  which  consumes  as  much  as  five 
or  ten  years  traversing  a  sequence  of  phases  which 
begins  with  a  feasibility  study  and  ends  with  final 
implementation.  It  is  not  uncommon  for  knowledge 
concerning  optimal  performance  to  become  known  so 
late  in  the  development  that  changes  in  many  design 
parameters  are  to  costly  to  make.  A  methodology 
is  badly  needed  which  will  permit  a  more  rapid  per¬ 
formance  of  scientific  experiments  on  information 
systems.  Such  a  methodology  will  not  only  increase 
progress  in  the  research  laboratory  but  if  applied 
early  in  the  development  cycle  will  also  provide  the 
necessary  feedback  to  permit  the  changes  in  design 
parameters  which  achieve  a  closer  to  optimal  perfor¬ 
mance  for  the  total  system.  It  is  the  thesis  of  this  paper 
that  computer  simulation  provides  a  powerful  meth¬ 
odology  suitable  for  attaining  these  objectives.  Up 
to  the  present  this  methodology  has  been  applied 
mostly  to  solving  hardware  design  problems.  For 
the  future  a  greater  emphasis  will  be  placed  on  includ¬ 
ing  software  control  subsystems  within  the  scope  of 
simulation  experiments.  This  is  particularly  important, 
for  example,  in  storage  and  retrieval  systems  which 
provide  on-line  service  to  a  collection  of  users  for 
the  logical  organization  of  system  control  is  sub¬ 
stantially  imbedded  in  software. 

Our  experience  has  indicated  that,  to  a  greater 
degree  than  is  provided  by  alternative  approaches, 
simulation 


•  provides  a  vivid  and  understandable  picture  of 
how  a  system  works, 

•  makes  explicit  those  assumptions  which  are 
often  implicit, 

•  encourages  the  use  of  explicit  measures  of 
performance, 

•  permits  decision-makers  to  play  an  active  role 
earlier  in  the  design  process, 

•  provides  a  convincing  demonstration  of  logical 
completeness  and  consistency, 

•  provides  decision  makers  with  greater  insight 
into  how  to  make  trade-offs  in  system  param¬ 
eters, 

•  inspires  confidence  in  management  and  results 
in  a  better  engineering  of  consent  among  those 
who  will  be  required  to  implement,  operate  and 
use  the  resulting  system, 

•  permits  an  early  recognition,  formulation  and 
solution  of  difficult  and  subtle  interface  prob¬ 
lems,  and 

•  provides  an  early  discovery  of  explicit  aspects 
of  the  environment  which  require  further  study. 

Basic  coticepts  in  cotnputer  simulation 

Simulation  is  a  problem  solving  technique.  Es¬ 
sentially  it  is  a  particular  technique  of  the  experi¬ 
mental  method  and  is  especially  useful  in  situations 
where  experimentation  with  the  physical  system  is 
inconvenient,  costly  or  impossible.  In  such  situations 
an  abstract  model  of  the  proposed  or  physical  system 
is  created  and  experiments  are  performed  upon  the 
abstract  model.  Experiments  with  models  of  informa¬ 
tion  systems  generally  involve  dynamical  relation¬ 
ships.  For  such  systems  simulation  may  be  defined 
briefly  as  a  dynamic  representation  achieved  by  con¬ 
structing  a  model  and  driving  it  through  time.  In 
principle  the  clerical,  arithmetic  and  decision-making 
operations  required  to  drive  a  model  can  be  performed 
by  hand.  In  practice  large  or  complex  models  require 
the  assistance  of  electronic  computers  to  carry  out 
the  enormous  number  of  operations.  The  logical 
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structure  of  the  model  and  the  control  mechanisms 
for  driving  it  are  incorporated  within  a  computer  pro¬ 
gram.  The  computer  program  generally  will  also  in¬ 
clude  suitable  controls  over  the  course  of  the  experi¬ 
ment  and  a  complete  record  of  the  internal  opera¬ 
tion  of  the  model.  The  program  not  only  generates  a 
variety  of  data  but  also  processes  data,  analyzes 
data  and  prepares  reports  to  integrate  the  results  of 
the  experiment.  The  general  procedure  for  implement¬ 
ing  a  computer  simulation  experiment  is  outlined  in 
Figure  I. 


Figure  1 — Procedure  for  simulation  experiment 


Implementation  of  simulation  experiments 

The  design  of  a  simulation  experiment,  as  in  other 
forms  of  experimentation,  should  depend  on  the  mo¬ 
tivation  and  objectives  of  the  experimenter.  The 
following  situations  may  occur  singly  or  in  combi¬ 
nation: 

•  The  experimenter  has  a  hypothesis  concerning 
a  system.  T  he  experiment  is  designed  to  confirm 
or  reject  the  hypothesis. 

•  The  experimenter  has  alternative  system  de¬ 


signs.  The  experiment  is  designed  to  select 
the  “best”  alternative. 

•  The  experimenter  has  a  proposed  system  de¬ 
sign.  The  experiment  is  designed  to  study  the 
behavior  and  performance  of  the  system. 

•  The  experimenter  has  a  description  of  a  sys¬ 
tem.  The  experiment  consists  in  modelling  and 
simulating  the  system  to  determine  whether 
the  description  is  logically  consistent  and 
complete. 

The  last  situation,  though  not  in  the  classical  pat¬ 
tern  for  an  experiment,  has  been  found  very  useful 
in  testing  preliminary  design  concepts.  With  a  modest 
outlay  of  effort  the  designer  can  quickly  spot  basic 
flaws  or  holes  in  the  design.  Frequently  these  are 
found  in  the  modeling  phases  even  before  the  com¬ 
puter  program  is  written.  In  the  design  of  complex 
systems  a  single  experiment  is  generally  inadequate 
to  solve  the  collection  of  problems  facing  the  de¬ 
signer.  It  then  becomes  necessary  to  lay  out  a  pro¬ 
gram  of  experimentation  and  to  modify  the  program 
as  required.  The  experimenter’s  skill  and  experience 
in  the  use  of  the  scientific  method  play  an  important 
role  in  the  successful  organization,  implementation 
and  execution  of  such  a  program.  Finally,  if  the  ex¬ 
perimenter  intends  to  make  a  contribution  to  knowl¬ 
edge  he  has  the  obligation  to  document  the  experi¬ 
ment  with  care  and  thoroughness  so  that  other 
independent  investigators  can  duplicate  the  experi¬ 
ment  and  confirm  the  results  and  conclusions. 

Defining  the  problem 

The  problem  definition  phase  is  crucial  to  the  design 
of  a  simulation  experiment.  A  poorly  defined  problem 
will  lead  to  an  inadequate  model  and  inconclusive 
results.  A  number  of  important  and  basic  questions 
are  posed  below  to  which  the  experimenter  should 
provide  satisfactory  answers  before  he  proceeds. 

•  What  problem  am  I  trying  to  solve?  If  the 
problem  is  solved  what  new  knowledge  will  be 
available?  How  important  is  this  knowledge? 
To  whom? 

•  Toward  what  objectives  should  the  solution  of 
the  problem  be  oriented? 

•  Am  I  solving  this  problem  in  behalf  of  a  de¬ 
cision-maker?  If  so,  does  he  agree  with  my 
statement  of  the  problem  and  the  objectives 
to  be  attained? 

•  What  form  of  solution  is  acceptable  to  me?  To 
the  decision-maker? 

In  system  design  research  problems  and  objectives 
have  a  tendency  to  change.  It  is  therefore  wise  for 
the  experimenter  to  periodically  return  to  the  above 
questions  and  probe  for  desirable  alterations  to 
previous  answers.  During  the  process  of  providing  a 
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complete  definition  of  the  problem  the  experimenter 
must  consider  the  following  factors: 

•  Information  known  or  assumed  about  the  sys¬ 
tem,  its  parts,  functions,  properties  and  relations. 

•  System  variables  which  affect  performance. 

•  Variables  in  the  environment  which  interact 
with  the  system. 

•  Variables  which  can  be  controlled. 

•  Variables  which  are  not  controlled  and  the 
mechanisms  by  which  their  values  arc  set. 

•  Measures  of  attainment  of  desired  objectives. 

It  is  usually  the  case  in  complex  information  sys¬ 
tems  to  present  several  measures  of  performance  so 
that  performance  may  be  represented  by  a  vector 

P  =  (M,  . Mn)  where  the  Mj  arc  the  various 

measures.  It  is  useful  whenever  possible  to  derive 

an  objective  function  V  =  f(Mi . MN)  which  yields 

an  overall  measure  of  system  performance.  Such 
an  objective  function  may  be  justified  mathematical¬ 
ly  or  offered  to  the  decision  maker  as  a  reasonable 
criterion  to  accept.  Since  the  Mj  are  presumed  to  be 
functions  of  relevant  system  variables  Vh  it  is  there¬ 
fore  reasonable  to  represent  the  objective  function 
by  V 

V  =  F(Vf . Vk, 

The  following  represents  some  of  the  ways  in  which 
an  optimization  problem  may  be  formulated  for  an 
experiment. 

•  It  may  be  known  or  tentatively  assumed  that 

there  exists  a  unique  vector  (V,+ . Vk*) 

which  yields  an  optimal  value  V*  for  the  ob¬ 
jective  function  F. 

•  The  objective  function  V  may  be  accompanied 
by  constraints.  For  example  we  may  require 
that  Mi  2s  Aj  where  A[  arc  prescribed  values. 

•  It  may  be  known  or  tentatively  assumed  that 
the  optimal  value  V*  is  not  attained  for  a  unique 
vector  and  that  the  equation  F(V,, .  .  .  ,Vk)=V* 
defines  an  “optimal  surface." 

The  experimenter  may  not  know  initially  how  to 
best  define  his  policy  of  optimization.  He  may  begin 
with  the  first  situation  above  and  find  that  the  opti¬ 
mum  solution  occurs  at  a  place  where  one  of  the  Mj 
is  disappointingly  low.  Thus  he  finds  that  the  objective 
function  above  is  not  a  completely  satisfactory 
criterion  and  he  is  led  to  formulate  the  next  situation. 
However,  he  discovers  that  the  optimum  is  no  longer 
unique  and  he  is  really  in  the  third  situation. 

Another  difficulty  which  frequently  arises  is  that 
the  experimenter  does  not  in  fact  know  how  the  Mj 
depend  on  the  system  variables  VV  He  must  therefore 
design  one  or  more  experiments  dedicated  to  the 
task  of  establishing  the  true  causal  relationships  and 
determining  the  functions  which  relate  the  Mj  to  the 


relevant  Vh  These  considerations  help  to  show  why 
the  experimenter  must  frequently  conduct  prelimi¬ 
nary  experiments  and  why  a  sequence  of  experi¬ 
ments  does  not  always  follow  a  predetermined  path. 

Modeling  considerations 

However  powerful  a  technique  computer  simula¬ 
tion  may  be  it  must  be  remembered  that  it  is  not  the 
only  approach  to  solving  problems  in  the  design  of 
information  systems.  The  experimenter  should  con¬ 
sider  all  possible  alternative  approaches  before  de¬ 
ciding  to  embark  on  simulation  experiments.  The 
analytic  approach  makes  use  of  mathematical  models 
and  mathematical  methods  of  solution.  This  approach, 
when  feasible,  is  frequently  more  economical  than 
other  approaches  and  provides  powerful  tools  for 
predicting  system  response  as  a  function  of  relevant 
system  parameters.  Mathematical  analysis  has  been 
difficult  to  apply  to  systems  which  involve  multiple 
interacting  queues  and  complicated  scheduling  rules. 
However,  progress  is  being  made  in  developing  mathe¬ 
matical  tools  in  the  areas  of  queuing  theory,  mathe¬ 
matical  programming  and  network  flows  which  will 
be  applicable  to  the  solution  of  more  difficult  prob¬ 
lems.  Significant  progress  will  be  made  when  research¬ 
ers  learn  how  to  effectively  combine  simulation  and 
analytic  techniques.  One  method  which  has  been 
successfully  employed  used  mathematical  analysis 
in  one  or  more  of  the  subsystems  and  applied  the  re¬ 
sults  to  simplify  the  simulation  of  the  total  system. 

In  construction  of  a  model  the  experimenter  is 
attempting  to  achieve  a  representation  which  at  one 
and  the  same  time  is  less  complicated  than  the  real 
system  and  easier  to  use  for  research  purposes.  He 
achieves  simplicity  by  representing  only  those  prop¬ 
erties  which  arc  relevant  to  the  solution  of  the  pro¬ 
blem  In  very  large  and  complicated  systems  the  ex¬ 
perimenter  has  to  find  a  good  balance  between  the  ac¬ 
curacy  of  representation  and  the  manageability  of 
the  model.  Practical  considerations  which  enter  here 
include  time,  cost,  personnel,  the  computer  to  be  used 
and  the  nature  of  the  programming  system.  A  variety 
of  simplification  techniques  exists  for  this  purpose 
and  the  experimenter  selects  from  among  them.  It 
is  important  for  the  experimenter  to  make  these  tech¬ 
niques  explicit  and  record  them  so  that  any  unsatis¬ 
factory  results  from  the  experiment  can  be  investi¬ 
gated  in  the  light  of  the  simplification  techniques 
employed.  In  achieving  the  desired  degree  of  approxi¬ 
mation  the  model  builder  may  use  two  different  ap¬ 
proaches: 

•  Start  with  the  simplest  version  possible  and 
“sneak  up"  on  the  final  model  by  adding  ap¬ 
propriate  refinements  until  satisfactory  results 
are  achieved. 
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•  Start  with  a  model  which  incorporates  all  the 
details  and  carve  away  the  least  relevant  fea¬ 
tures  one  at  a  time  checking  to  see  that  satis¬ 
factory  results  can  still  be  achieved. 

The  author  favors  the  first  approach  for  the  fol¬ 
lowing  reasons: 

•  It  is  easier  to  begin  with  simple  situations  and 
permit  the  model  builder  to  “warm  up.” 

•  The  experimenter  receives  earlier  results  and 
may  use  them  to  sharpen  his  understanding  of 
the  system’s  behavior.  The  feedback  is  useful 
in  further  modeling. 

•  It  is  easier  to  implement  because  computer 
programs  are  more  easily  debugged  in  stages. 

•  It  is  easier  to  analyze  results  and  evaluate  the 
experiment  with  a  simpler  model. 

•  It  appears  to  be  a  more  economical  way  to  arrive 
at  the  goal. 

•  It  simplifies  project  management  problems. 

Finally,  we  must  acknowledge  that  there  exists 

no  firm  procedure  by  means  of  which  the  experi¬ 
menter  can  construct  a  satisfactory  model.  Model¬ 
building  remains  a  highly-sophisticated  engineering 
art  and  its  successful  practice  depends  on  the  skill, 
judgment  and  experience  of  the  experimenter. 

Validating  the  model 

Validation  procedures  are  employed  by  the  experi¬ 
menter  to  answer  the  following  questions: 

•  Does  the  model  function  as  it  was  designed  to 
function? 

•  Docs  the  model  yield  simulation  histories 
which  are  valid  representations  of  the  system 
under  study? 

The  first  question,  perhaps  more  properly  referred 
to  as  model  testing,  is  relatively  easier  than  the  sec¬ 
ond.  The  distinction  between  the  two  types  of  ques¬ 
tions  can  be  illustrated  with  a  simple  example.  Sup¬ 
pose  the  model  is  designed  to  represent  a  computer 
system  in  which  jobs  to  be  executed  by  the  computer 
have  different  priorities.  Suppose  further  that  the 
model  is  designed  so  that  50  per  cent  of  the  jobs  which 
arrive  are  priority  1  jobs.  To  answer  the  first  question 
the  experimenter  checks  to  see  if  the  model  gener¬ 
ates  job  arrivals  in  such  a  way  that  50  per  cent  of 
them  are  priority  1  jobs.  To  answer  the  next  question 
the  experimenter  must  show  that  50  per  cent  of  the 
jobs  which  arrive  at  the  real  system  are  in  fact  priority 
1  jobs.  In  this  example  model  testing  and  model 
validation  can  employ  similar  statistical  techniques. 
However,  the  application  of  these  techniques  to  the 
two  cases  will  confront  the  experimenter  with  very 
different  problems,  particularly  with  respect  to  the 
problem  of  data  acquisition. 


In  model  testing  the  data  required  is  obtained  from 
simulation  runs  and  specifications  of  the  model.  The 
experimenter  will  check  to  see  that  the  results  from 
the  simulation  runs  reflect  suitably  on  the  assumptions 
and  properites  in  the  design  of  the  model.  The  relevant 
variables  are  checked  to  see  if  they  lie  in  the  pre¬ 
scribed  or  predictable  ranges.  The  statistical  variables 
are  checked  for  bias  and  extremes  in  variation  from 
the  mean.  Further  statistical  tests  may  be  required 
to  verify  that  the  random  variables  are  distributed  in 
accordance  with  prescribed  statistical  distributions. 
The  degree  of  reliability  which  is  associated  with 
such  statistical  testing  will  usually  have  implications 
with  respect  to  the  sampling  technique  and  size  of 
sample.  In  turn,  the  size  of  sample  will  have  implica¬ 
tions  on  the  duration  of  the  simulation  run.  In  such 
matters  the  experimenter  should  be  competent  in  the 
area  of  statistical  testing  of  hypotheses  or  be  pre¬ 
pared  to  seek  such  advice.  In  gathering  data  from  the 
model  the  experimenter  should  avoid  using  data  gen¬ 
erated  during  initial  transient  conditions  of  the  model. 
Sometimes  the  transient  conditions  can  be  eliminated 
by  suitably  initiating  the  model.  Other  techniques 
exist  for  reducing  the  duration  of  the  transient  state 
of  the  model.  It  is  generally  a  good  idea  to  make  a 
number  of  runs  with  different  values  of  the  model 
parameters  to  insure  the  stability  of  the  model  over 
the  parameter  space  which  will  be  of  interest  to  the 
experimenter. 

In  validating  their  models  experimenters  face  two 
kinds  of  situations: 

•  The  model  was  built  to  represent  a  real  system 
w  hich  can  be  observed  in  operation. 

•  The  model  was  built  to  represent  a  hypothetical 
system  which  cannot  be  observed  in  operation. 

For  the  first  situation  the  task  of  validation  is  rela¬ 
tively  simple.  The  basic  approach  is  to  compare  the 
behavior  of  the  model  with  the  behavior  of  the  real 
system.  Historical  data  collected  from  the  real  system 
will  be  very  useful  in  making  such  a  comparative 
analysis.  Difficulties  may  be  encountered  if  the  his¬ 
torical  data  is  incomplete  or  unreliable.  A  more 
crucial  test  is  to  use  the  predictive  capabilities  of  the 
model  to  forecast  the  behavior  of  the  system.  Here 
the  experimenter  can  expect  difficulties  in  gathering 
data,  especially  if  the  process  of  acquiring  the  data 
tends  to  interfere  with  the  normal  operation  of  the 
system. 

In  the  second  situation,  which  is  the  one  in  which 
the  system  designer  generally  finds  himself,  the  prob¬ 
lem  of  validation  is  most  difficult.  The  model’s  be¬ 
havior  cannot  be  compared  with  an  actual  system  but 
perhaps  some  of  its  subsystems  can  be  so  compared. 
And  perhaps  there  are  similar  systems  with  which  it 
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may  be  partially  compared.  In  many  instances  the 
designer  has  to  settle  for  the  reasonableness  of  the 
model  and  the  degree  to  which  its  behavior  conforms 
to  his  own  intuitive  understanding  of  such  a  system. 
Even  so  the  model  may  be  efficient  for  the  purpose 
of  making  relative  comparisons  of  system  behavior 
with  respect  to  a  selection  of  alternatives  such  as 
decision  rules.  In  obtaining  relative  results  simulation 
offers  a  special  advantage  in  that  all  the  variables 
can  be  controlled  so  that  the  same  sequence  of  system 
inputs  can  be  repeated  in  each  run.  Even  the  random 
events  can  be  made  to  repeat  so  that  the  variations 
in  the  system  output  cannot  be  attributed  to  random 
fluctuations  between  runs. 

Preparation  of  the  computer  program 

The  design  of  a  computer  simulation  experiment 
must  interweave  into  a  harmonious  pattern  the  de¬ 
sign  of  a  scientific  experiment  and  the  design  of  a 
computer  program.  In  order  to  obtain  maximum 
service  from  the  computer  the  experimenter  will 
incorporate  into  the  program: 

•  The  structure  of  the  model, 

•  the  logical  machinery  to  drive  the  model, 

•  analytic  and  decision  making  procedures, 

•  the  procedures  which  generate  and  record 
system  status  information,  and 

•  the  routines  which  assemble,  process  and  ar¬ 
range  the  data  into  final  output  reports. 

The  experimenter  frequently  encounters  two  kinds 
of  difficulties: 

•  Exceeding  eorc  storage. 

•  Excessive  run  time. 

The  causes  of  these  difficulties  are  various  and  the 
experienced  designer  learns  to  anticipate  these  prob¬ 
lems  in  the  earlier  stages  of  designing  the  experiment. 
In  considering  these  problems  the  designer  will  often 
judge  the  desirability  of: 

•  Reducing  the  size  of  the  experiment, 

•  simplifying  the  model, 

•  segmenting  the  computer  program,  and 

•  different  computers  and  programming  systems. 

Exceeding  core  storage 

Large  and  complicated  models  will  naturally  occupy 
a  large  amount  of  storage.  When  such  a  model  is 
driven  the  data  generated  by  the  model  will  some¬ 
times  overflow  memory.  When  simplification  of  the 
model  cannot  solve  the  problem  the  experimenter 
may  consider  running  the  program  in  distinct  se¬ 
quential  phases,  where  each  phase  occupies  available 
core  during  its  execution.  The  following  breakdown 
has  often  worked  successfully  : 


•  Input  phase  —  Initial  system  status  is  read  in, 
tables  to  generate  random  variates  are  con¬ 
structed  and  exogenous  events  are  generated 
and  stored  on  magnetic  tapes. 

•  Simulation  phase  —  The  model  is  driven  using 
the  exogenous  event  tape  as  input.  System 
status  information  is  generated  and  stored  on 
tape. 

•  Output  phase  —  The  system  status  information 
is  input,  processed,  and  analyzed  to  produce 
output  reports. 

Another  way  to  avoid  memory  overflow  is  to  re¬ 
duce  the  duration  of  simulation.  For  example  instead 
of  simulating  the  operation  of  a  computer  system  24 
hours  it  may  be  possible  to  simulate  three  8-hour 
shifts  in  three  separate  computer  runs.  With  this 
approach  provision  must  be  made  to  preserve  con¬ 
tinuity,  i.e.,  the  system  status  at  the  beginning  of 
a  next  shift  must  be  the  same  as  the  system  status 
at  the  termination  of  the  preceding  shift. 

Excessive  run  time 

A  computer  simulation  experiment  may  sometimes 
make  excessive  demands  for  computer  time.  The 
most  frequent  causes  for  this  are: 

•  Large  decision  space. 

•  Unfavorable  time-compression  ratio. 

•  Slow  stochastic  convergence. 

Large  decision  space 

Let  the  function 

V  =  F  (U„...,UJ  V„...,Vn) 
represent  an  objective  function  in  which  the  U, 
represent  uncontrolled  system  variables  and  the  Vj 
represent  the  controlled  or  decision  variables.  If 
the  designer  has  an  optimization  problem  to  solve, 
i.e.,  to  find  values  for  the  V,  such  that  V  attains  its 
optimum  value  an  obvious  approach  may  be  to  rep¬ 
licate  the  simulation  experiment  as  many  times  as 
the  cardinal  number  of  the  decision  space,  i.e., 
the  number  of  choices  for  the  decision  vector  (V,,..., 
Vn).  With  even  a  small  number  of  variables  V,  and 
a  range  of  10  different  values  for  each  variable  the 
number  of  replications  may  become  prohibitive. 
When  this  happens  the  experimenter  may  have  to 
arbitrarily  reduce  the  size  of  the  decision  space 
or  use  a  more  sophisticated  experimental  design  such 
as  a  fractional  factorial  design.  Other  possibilities 
involve  simplification  of  the  model  in  one  way  or 
another. 

Unfavorable  time-compression  ratio 

If  the  computer  program  is  executed  in  t  units 
of  real  time  and  drives  the  model  through  T  units 
of  simulated  time  then  the  ratio  Tit  is  called  the  time- 
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compression  ratio  for  the  program.  A  computer  pro¬ 
gram  which  simulates  an  information  processing 
system  and  which  has  a  time-compression  ratio  of 
100:1  can  provide  a  simulation  of  24  hours  of  pro¬ 
cessing  in  about  15  minutes  on  the  computer  which 
is  quite  reasonable  in  cost.  Consider  now  a  model 
in  which  certain  events  occur  every  few  milliseconds 
and  other  events  occur  say  every  few  minutes.  There 
is  contained  in  the  same  model  a  micro-world  of  events 
and  a  macro-world  of  events  and  the  time-compres¬ 
sion  ratio  for  the  model  will  be  governed  by  the  time- 
compression  ratio  for  the  micro-world  which  in  such 
situations  is  likely  to  be  quite  small,  say  1 : 1  or  smaller. 
To  simulate  an  8-hour  shift  with  such  a  time-compres¬ 
sion  ratio  will  be  very  costly.  One  approach  to  con¬ 
quering  this  difficulty  lies  in  simulating  the  micro¬ 
world  of  events  separately  and  to  characterize  the 
behavior  of  the  micro  system  so  that  its  effect  can 
be  imbedded  in  the  macro  system  at  the  macro-level. 
Other  approaches  have  been  used  which  avoid  dealing 
explicitly  with  the  micro-level  (Katz,  1966). 

Slow  stochastic  convergence 

Information  systems  which  operate  as  service  sys¬ 
tems  generally  contain  queuing  subsystems.  Models 
which  contain  stochastic  subsystems  generally  pass 
through  transient  phases  before  reaching  a  “steady 
state.”  When  a  model  converges  to  the  “steady  state” 
very  slowly  an  excessive  amount  of  computer  time  is 
used.  Various  techniques  have  been  adopted  to  avoid 
the  long  transient  period  during  simulation.  When 
approximate  information  can  be  obtained  about  the 
steady  state  the  model  is  initialized  so  that  it  starts 
with  an  initial  state  relatively  near  to  the  steady  state. 
For  example,  when  queues  are  involved  these  are 
pre-loaded  with  customers  when  simulation  starts. 

Run  times  are  frequently  governed  by  the  statistical 
sampling  techniques  employed  to  obtain  estimates  of 
random  variables.  Excessive  run  time  may  sometimes 
be  reduced  by  using  more  efficient  sampling  tech¬ 
niques  and  more  sophisticated  statistical  analysis. 

Progress  in  computer  simulation 

The  last  few  years  have  seen  a  rapid  growth  in  the 
application  of  computer  simulation  to  the  design  and 
development  of  new  systems.  The  accumulated  ex¬ 
perience  and  successes  in  these  projects  have  in¬ 
spired  the  development  of  a  number  of  simulation 
languages  for  programming  simulation  applications. 
Among  these  GPSS  and  S1MSCR1PT  have  attracted 
a  considerable  number  of  users.  Improvements  have 
been  made  in  both  of  these  languages  and  plans  are 
under  way  to  develop  new  language  systems  which 
will  provide  even  more  powerful  tools  for  the  experi¬ 


menter.  Research  effort  at  IBM  and  at  Lockheed,  for 
example,  has  been  expended  to  develop  languages 
which  are  “process  oriented”  rather  than  “event 
oriented.”  One  such  language  system  developed  at 
Lockheed  has  recently  been  implemented  and  is  now 
ready  for  use  in  the  field. 

The  system  designer  and  the  system  analyst  are 
both  concerned  with  two  important  questions  about  a 
particular  system: 

•  How  can  I  accurately  describe  the  system? 

•  How  can  I  evaluate  its  performance? 

It  is  clear  that  progress  in  systems  theory  should 
have  important  consequences  for  applications  like 
system  design  and  system  analysis.  A  general  theory 
of  systems  will  undoubtedly  imply  a  general  theory  of 
models  and  this  should  lead  to  important  conse¬ 
quences  for  the  model  builder.  System  theory  is  young 
and  it  lies  in  the  province  of  basic  research.  For  the 
present  its  importance  relates  more  to  future  promise 
than  to  past  accomplishment.  The  applied  scientist 
should  follow  its  developments  and  be  prepared  to  test 
theoretical  concepts  and  evaluate  them  for  their  po¬ 
tential  utility  in  system  applications. 
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“Anyway,  the  actual  workings  of  the  Secret 
Service,  like  those  of  criminal  investigation,  hold 
a  lintited  emotional  appeal  for  most  people 

KINGSLEY  AMIS  in  The  James  Bond  Dossier 

PROLOGUE 

The  zoo  and  the  jungle 

The  naturalist  must  always  be  careful  to  distinguish 
between  zoo  behavior — the  behavior  of  animals  in  cap¬ 
tivity — and  the  behavior  of  the  same  species  in  the 
wild  state.  He  should  also  be  continually  aware  of  the 
effect  of  the  observation  upon  the  phenomenon  ob¬ 
served. 

In  many  ways — surprisingly  enough,  most  of  them 
good — the  relationship  of  the  intelligence  analyst  to 
his  supporting  information  systems  may  be  regarded  as 
zoo  behavior.  He  lives  on  the  same  premises;  simple 
operant  conditioning  of  both  the  analyst  and  the  in¬ 
formation  system  can  bring  the  two  into  harmony.  If 
he  withers  from  lack,  or  bloats  from  excess,  of  ade¬ 
quate  information  the  zoo-keeper  has  certain  embar¬ 
rassing  immediate  problems  to  face. 

The  scientist  in  the  state  of  nature  is  another  mat¬ 
ter  entirely.  He  has  to  be  lured  to  the  information 
system  with  salt  blocks,  strange  scents  and  even  more 
peculiar  calls.  The  observer  in  his  camouflaged  blind 
sees  only  healthy,  active  specimens.  By  definition,  the 
inactive  are  not  observed,  but  can  only  be  inferred. 

INTRODUCTION 

This  paper  is  an  attempt  to  explore  and  contrast  two 
related  areas — the  information  requirements  of  the  in¬ 
telligence  analyst  and  the  practicing  scientist.  “Intel¬ 
ligence”  is  used  in  this  paper  in  a  highly  restricted 
sense.  It  is  limited  almost  entirely  to  that  information 
gleaned  from  open  scientific  and  technical  literature — 
difficultly  accessible,  perhaps,  but  at  least  in  its  country 
of  origin,  available  to  the  nationals  of  that  country. 


Sherman  Kent  (Kent,  1965)  has  pointed  out  that 
there  arc  areas  of  the  world  in  which  the  phrase  “overt 
intelligence”  would  be  regarded  as  an  oxymoron  For, 
as  he  writes: 

“If  in  fact  the  Soviets  engage  in  what  we  of  the 
West  call  ‘ intelligence  research  ami  analysis *  they 
have  another  name  for  it  and  a  name  bereft  of 
the  cachet  of  ‘intelligence.’  It  is  seemingly  in¬ 
conceivable  to  them  that  large  numbers  of  people 
will  be  quite  overtly  engaged  in  something  known 
as  intelligence  work,  able  to  inform  all  and  sundry 
that  this  is  in  fact  their  calling ,  and  obliged  to 
guard  with  secrecy  only  those  matters  having  to 
do  with  their  sources ,  methods ,  the  foci  of  their 
attention  and  the  content  of  their  findings’* 

Or,  to  quote  a  Russian  source  (Orlov,  1963)  : 

“According  to  the  views  of  Russian  officers,  it 
takes  a  man  to  do  the  creative  and  highly  dan¬ 
gerous  work  of  underground  intelligence  on  foreign 
soil;  as  to  the  digging  up  of  research  data  in  the 
safety  of  the  home  office  or  library,  this  can  be 
left  to  women  or  young  lieutenants  who  have  just 
begun  intelligence  careers.*’ 

Or  again,  to  quote  Gen.  William  Donovan  (cited  in 
Orlov,  ibid.): 

“Intelligence  is  not  the  mysterious,  even  sinis¬ 
ter  thing  people  think  it  is,  but  is  more  a  pattern 
of  pulling  together  myriad  facts,  making  a  pat¬ 
tern  of  them,  and  drawing  inferences  from  that 
pattern.'* 

This  is  the  intelligence  pattern  ( razvedka  or  no)  to 
be  discussed  in  this  paper. 

The  information  practices  of  scientists 

Study  of  the  information  habits  of  scientists  has 
almost  become  the  dermatology  (the  patient  usually 
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does  not  die,  and  he  never  really  gets  well)*  of  the 
information  sciences.  One  does  occasionally  hear  of 
the  experiments  which  fail,  as  in  the  letter  in  Science 
(Dray,  1966)  recommending  that  the  Information 
Exchange  Group  on  Immunopathology  be  discontinued 
for  nine  apparently  valid  reasons.  Failure  to  meet 
user  needs  is  at  least  one  of  the  many  reasons  why 
information  systems  totter  and  fall  (see  the  author’s 
Post-Mortems  Can  Be  Fun)  (Wooster,  1965a),  but 
studies  of  user  requirements  seem  to  go  on  forever. 

The  International  Conference  on  Scientific  Informa¬ 
tion,  held  in  Washington  in  1958,  provides  a  con¬ 
venient  starting  point  to  survey  the  literature  in  the 
field.  Tornudd  reviewed  most  of  the  information  use 
studies  published  before  1958.  Her  bibliography  cites 
69  articles.  (Tornudd,  1958) 

That  same  year  Mortimer  Taube  (Taube,  1958)  at¬ 
tempted  an  evaluation  of  the  then  total  existing  litera¬ 
ture  of  use  studies,  first  pointing  out  that  of  the  69 
studies  cited  by  Tornudd  only  15,  dating  from  1955, 
had  not  been  listed  in  previous  compilations  by  Shaw 
(Shaw,  1956),  Henkle  (Henkle,  1956)  and  Stevens 
(Stevens,  1953)  dating  back  to  1953.  Taube  added  to 
the  69  Tornudd  citations  the  12  papers  on  the  subjeet 
given  at  the  Conference  itself,  bringing  the  total  to  81. 

Menzel,  in  1960  (Menzel,  1960)  attempted  to  col¬ 
late  tables  of  findings  from  more  or  less  comparable 
studies.  His  bibliography,  admittedly  more  exclusive 
than  those  cited  above,  lists  26  articles. 

In  1964,  Davis  and  Bailey  (Davis  and  Bailey,  1964) 
compiled  an  annotated  bibliography  of  438  “use 
studies”  containing  virtually  every  study  of  significance 
up  to  1963.  One  reviewer  (Paisley,  1965)  has  pointed 
out  that  “This  is  a  difficult  source  to  use  because  of 
the  high  proportion  of  chaff;  about  350  of  the  citations 
are  commercial  periodical  readership  studies  and  li¬ 
brary  school  research  exercises,”  which  would  seem 
to  leave  a  total  of  188  valid  papers. 

This  same  reviewer  (Paisley,  Idem)  cited  some  75 
relevant  papers  in  his  review  of  the  literature. 

Herner  and  Company  in  1966  (Herner  and  Co., 
1966)  broadening  the  scope  to  include  physicians, 
were  able  to  locate  “several  hundred”  papers  dealing 

♦Since  the  disease  of  Venus  are,  at  the  moment,  of  more 
interest  to  the  dermatologist  than,  one  would  hope,  lo  either 
the  information  specialist  or  the  astronaut,  the  author's  bio¬ 
medical  background  compels  him  to  warn  of: 

*\  .  .  the  young  man  from  Bombay 
Who  thought  lues  just  went  away 
As  a  result  he  has  tabes, 

And  bandy-legged  babies 

And  thinks  he  is  Queen  of  the  May." 

An  inaccurate  version  of  this  medical  mnemonic  may  be  found 
in  (Gordon,  1957). 


with  information  patterns  in  science,  with  110  dealing 
specifically  with  the  problem  in  biomedical  sciences. 

Some  idea  of  the  flavor  of  the  field  may  be  ob¬ 
tained  from  two  papers.  The  first  is  the  “classical” 
1958  paper  (the  author  is  tempted  to  define  classical 
as  having  been  around  long  enough  to  pass  into  the 
Russian  literature  and  then  back  out  again  via  an 
English  journal,  the  original  source  being  lost  in  the 
process)  of  Russell  Aekoff,  then  heading  the  Opera¬ 
tions  Research  Group  at  Case  Institute  of  Technology 
(Aekoff,  1958). 

Aekoff  et  al.  made  25,000  observations  on  more 
than  1,500  chemists  chosen  as  a  representative  sample 
of  the  U.S.  in  1957-1958.  Out  of  a  90-hour  week  (in¬ 
cluding  week-ends  and  evenings)  the  average  chemist 
spends  16.5  hours  in  scientific  communication,  10.4 
hours  with  equipment,  6.7  hours  in  business  communi¬ 
cation,  three  hours  in  data  treatment  and  all  of  two 
and  one-half  hours  in  thinking  and  planning.  The  aver¬ 
age  chemist  spends  more  time  in  eommunieating  (23.2 
hours  a  week)  than  in  all  the  rest  of  his  professional 
activities  combined  (15.9  hours  per  week). 

More  recently  the  “Auerbach  study”  (Auerbach 
Corp.,  1965)  which  lasted  16  months  and  cost  almost 
$300,000  attempted  to  discover  just  how  the  36,000 
scientists  and  engineers  in  the  Department  of  Defense 
actually  got  the  information  they  needed  to  do  their 
jobs.  They  found  that  half  of  these  either  consulted 
each  other  or  went  to  individuals’  personal  files  of 
information  to  meet  their  primary  needs.  In  39  per 
cent  of  these  eases  the  particular  information  require¬ 
ments  were  completely  satisfied. 

Equally  discouraging,  only  half  of  the  individuals 
interviewed  knew  of  their  own  information  services, 
which  were  rarely  ever  used  as  a  first  souree  of  in¬ 
formation.  About  one-fifth  of  them  had  never  heard 
of  the  Defense  Documentation  Center  (or  its  predeces¬ 
sor,  ASTIA);  nineteen  per  cent  didn’t  know  of  the 
existence  of  any  of  33  specialized  Department  of 
Defense  information  centers! 

The  job  of  the  intelligence  analyst 

In  contrast  to  the  almost  embarrassing  wealth  of 
information  about  the  information  requirements,  uses 
and  practices  of  scientists,  the  author  has  been  able 
to  find  only  one  unclassified  paper  relating  specifically 
to  the  information  needs  of  intelligence  analysts. 

In  fairly  precise  terms,  much  of  the  work  of  the 
intelligence  analyst  consists  of  the  beneficiation  of 
fairly  low  grade  ore,  or  even  tailings.  For  those  of  you 
who  did  not  grow  up  in  a  mining  community,  “bene- 
fieiation”  means  to  make  richer;  “tailings”  are  the  resi¬ 
due  from  previous  inefficient  refining  operations. 
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To  continue  the  mineralogical  metaphor  a  little  fur¬ 
ther,  the  sort  of  intelligence  operation  which  makes 
the  plot  for  best-sellers  (cither  as  paperbacks  or  pro¬ 
posals)  could  be  defined  as  “high-grading” — “to  steal 
rich  ore  from  a  mine,  especially  very  rich  gold  ore.” 
High-grading  was  very  mueh  a  matter  of  individual 
enterprise — an  individual  with  a  lenient  foreman  could 
stagger  out  of  a  mine  with  several  weeks’  pay  in  his 
pants  cuffs.  Reworking  tailings  is  a  much  duller  factory 
process,  calling  for  efficient  high  volume  machinery, 
figuring  profits  in  pennies  per  ton,  instead  of  dollars 
per  ounce. 

So  volume  is  the  first  item  to  keep  in  mind.  One 
information  installation  alone,  CIRC — Centralized  In¬ 
formation  Reference  and  Control,  described  by  Ray 
Barrett  (Barrett,  1965)  is  designed  to  process  about 
8,000  to  10,000  input  reports  per  month,  provide  a 
retrieval  capability  for  the  total  system  holdings,  and 
carry  out  weekly  dissemination  to  a  large  number  of 
user  groups.  By  way  of  contrast,  the  Clearinghouse  for 
Federal  Scientific  and  Technical  Information,  respon¬ 
sible  for  the  sale  of  U.S.  Government  sponsored  R&D 
reports  to  the  general  scientific  public,  adds  about 
2,300  reports  per  month,  plus  an  equal  number  of 
translations  (Fry,  1966). 

These  reports  arc  then  indexed,  classified,  abstracted, 
translated,  stored  and  retrieved.  These  may  or  may 
not  be  treated  as  conventional  library  processes,  with 
one  major  exception — the  assignment  of  a  subject 
classification  according  to  one  of  several  allowable 
hierarchical  coding  systems  which  have  become  more 
or  less  standard  throughout  the  intelligence  community. 
Amusingly  enough,  at  least  for  one  subject  area  in 
one  agency,  the  European  Universal  Decimal  Classifica¬ 
tion,  an  expatriated  form  of  Dewey  Decimal  Classifica¬ 
tion,  is  used.  Not  for  its  merits  which,  with  a  few 
isolated  exceptions  such  as  the  Engineering  Societies 
Library,  have  been  insufficient  to  overcome  the  chau¬ 
vinism  of  American  librarians  but  because  Russia,  in 
common  with  most  European  countries,  has  standard¬ 
ized  on  UDC  to  the  extent  of  printing  UDC  classifica¬ 
tion  numbers  on  journal  articles.  It  turns  out  to  be 
simpler  to  use  the  Russian  classification  system  in  this 
particular  instance  than  to  reclassify  the  articles. 

Presumably  a  substantial  portion  of  an  agency’s 
manpower  resources  arc  devoted  to  these  more  or  less 
routine  library  operations.  TDCK,  the  Netherlands 
Armed  Forces  Technical  Document  Center  in  The 
Hague,  which  resembles  an  intelligence  agency  at  least 
to  the  extent  that  its  preferred  output  is  evaluated  en¬ 
gineering  reports  rather  than  bibliographies,  devotes 
35  of  its  65  manpower  spaces  to  maintaining  the 
library  (Schuller,  1965).  The  Defense  Doeumcntation 
Center,  which  performs  solely  library  operations  in  the 


sense  of  this  paragraph,  uses  some  400  manpower  spac¬ 
es  to  process  and  retrieve  perhaps  3,000  to  4,000  re- 
per  month. 

Those  of  us  concerned  with  the  processing  of  scien¬ 
tific  information  would  ecrtainly  contend  that  money 
spent  in  improving  these  operations  is  “leverage” 
money,  that  saving  money  on  library  operations  is  false 
economy,  but  I  can  certainly  visualize  some  jim-dandy 
arguments  on  the  score  of  analysts  vs.  librarians. 

All  of  this  has  gone  on,  mind  you,  before  we  or  the 
documents  ever  meet  the  intelligence  analyst.  The 
analyst  sits  at  a  desk  and  reads  and  writes.  He  has  two 
major  sorts  of  jobs;  keeping  au  courant  in  a  particular 
subject/geographical/language  field — the  breadth  of  the 
field  varying  directly  with  the  grade  level  and  the  depth 
inversely — and  preparing  reports,  known  variously  as 
“finished  intelligence”  and  “evaluated  intelligence.” 

These  reports  fall  into  three  main  classes: 

Ci  ass  Synonyms 

PAST 

Basic  descriptive  Basic  research 

Fundamental  research 
Basic  data 
Monographic  data 
Encyclopedic  data 

PRESENT 

Current  reportorial  Current  intelligence 

Current  cvalualions 
Current  appreciations 
Cable  material 
Hot  intelligence 
FUTURE 

Speculate  ve-evaluati  ve  Esl  i  mates 

Strategic  estimates 
Evaluations 
Staff  intelligence 
Capabilities  intelligence 

(Kent,  1965) 

The  time  allowed  for  preparation  of  these  reports 
ranges,  as  one  would  expect,  from  the  near-academic 
to  the  nearer-frantic. 

The  information  from  which  these  reports  are  pre¬ 
pared  usually  conies  from  three  sources:  the  personal 
memory  of  the  analyst  (a  practice  facilitated  by  the 
tendency  to  minimize  the  usual  scholarly  apparatus  of 
citations  and  foot-notes  in  intelligence  reports);  what¬ 
ever  personal  files  the  analyst  has  looted  from  the  input 
flowing  across  his  desk  and  squirreled  away  against  a 
time  of  need  or  obtained  by  “unsystematically  canvass¬ 
ing  outside  sources  of  information”  (Kent,  1965),  and 
by  subject  searches  of  a  central  information  file. 

These  subject  searches  range  from  the  highly  specific 
— “What  was  the  rainfall  in  such  and  such  a  place  on 
such  and  such  a  date?”  to  perhaps  such  general  areas 
as  metals  and  stress  and  cryogenics.  The  analyst  differs 
slightly  from  the  normal  user  in  a  greater  tendency 
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to  use  such  qualifying  terms  as  “I  don’t  want  anything 
older  than  1958”  or  ‘i  don’t  want  documents  concern¬ 
ing  American  research”  or  he  may  even  say  “Docu¬ 
ments  must  all  be  unclassified.” 

I  have  at  times  had  the  impression  that  analysts  tend 
to  ask  very  broad  general  questions,  preferring  to  run 
barefoot  through  the  windrows  of  paper  in  search  of 
the  needle  of  truth  rather  than  trusting  the  information/ 
system  library  to  use  a  magnet.  Tastes  differ,  of  course. 
Some  analysts  say  that  they  never  use  a  subject  search, 
since  they  know  perfectly  well  what  data  are  available 
in  their  areas!  Others  prefer  to  frame  their  searches  in 
terms  of  authors,  geographical  locations,  facilities  and 
personalities.  There  arc  even  those  who  have  learned 
to  use  the  indexing  system  itself  as  a  rather  sensitive 


indicator,  spotting  trends  by  unexpected  increases  or 
decreases  in  the  use  of  certain  indexing  terms. 

At  last-Conjrontation! 

It  is  plausible,  if  rash,  to  attempt  to  compare  the 
information  practices  of  intelligence  analysts  and  scien¬ 
tists  in  the  following  table,  remembering  that  the  sep¬ 
aration  is  nowhere  near  as  clean  as  the  device  of  two 
columns  would  indicate  and  that  in  fact  one  class  of 
scientist — there  is  no  convenient  American  term  for 
this  class,  but  the  English  one,  “information  scientist” 
should  suffice — is  almost  indistinguishable  in  his  work 
habits  and  information  needs  from  the  intelligence 
analyst. 


Designation 

Use  of  oral/informal  information 


Table  I 

Intelligence  Analyst 

Seldom,  outside  his 
own  agency 


Scientist 

Preferred  mode  of  obtaining  current 
information.  Auerbach,  (1965) 


Attendance  at  open  scientific 

Usually  vicarious 

Avidly  (Korchin  &  Clarke,  1959) 

meetings 

Use  of  library 

Heavy 

Light 

Preferred  place  of  reading 

Desk 

Office,  laboratory  or  home — 
very  little  in  library 

Percentage  of  time  spent  in 

>75% 

<50%  (e.g.,  Aekoff  1958) 

information  processing 

Languages 

English  4-  (1-4) 

English  r h  0.5 

Prefer  translations 

Usually 

Always 

Use  of  foreign  sources  of  information 

Up  to  100% 

Less  than  2%  (Syracuse  U., 

1966) 

Maintain  personal  file 

Usually,  in  office 

17-100% — in  home  or  office 
(Jahoda,  1966) 

Good  Housekeeping  Seal  of 

Approval  for  literature 

TOP  SECRET 

Publication  in  referred  journal 
(Pasternack,  1966) 

Use  of  microfilm  or  microfiche 

Readily 

Only  in  desperation 

Leads  to  information 

Reference  services 

Gossip,  hot  tips  from  friends, 

scanni 

Bibliographic  sophistication,  e.g., 
ability  to  use  standard  reference 
tools 

Use  of  mechanized  current  awareness 
services,  e.g.,  KWIC  and  SDI 

Use  of  extra-mural  information 
sources,  e.g.,  specialized  informa¬ 
tion  centers 


High 


Good,  if  available 


Good 


(not  reading)  5-10  current  journals. 
Little  use  of  abstracting  services  (Ge¬ 
rard,  1958) 

Negligible  to  fair 


Good  if  chemists,  biologists,  ACM,  and/ 
or  NASA  contractors.  Otherwise,  fair 
to  poor.  (Sprague,  1965) 

Good  if  in  in-group;  otherwise  only  if 
led  gently  by  the  hand  or  touted  by  a 
buddy. 
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Table  I,  continued 


Designatum 

Intelligence  Analyst 

Scientist 

Number  of  full-seale,  retrospective 

1-12 

0-2 

literature  searches  per  year 

Availability  of  mechanized,  auto¬ 
mated,  computer-based,  etc. 
information  systems: 

Past 

Poor 

Zero 

Present 

Fair 

Poor 

Future 

Good 

Fair 

Principal  product 

Reports 

“New  scientific  knowledge”:  gossip; 
journal  articles  to  maintain  status; 
reports  to  maintain  sponsor. 

Attitude  toward  writing 

Evil  necessity 

Necessary  evil 

Ambienee — feed  baek  on  produets 

Ancchoic 

Resonant 

Output  processing  methods 

Early  Bronze  Age 

Late  Stone  Age 

Barriers  between  author  and 

3  to  10 

1  to  2;  journal  referees  and/or  editor. 

ultimate  consumer 

Average  cost  of  information 
services  per  user 

$200  to  $2,000  and 

$10  to  $200.  In  certain  rare  instances, 
e.g.,  pharmaeeutieal  firms,  as  high  as 

$1,000. 


Discussion  of  Table  / 

I  would  hope  that  the  high  face  validity — which  is 
a  fancy  word  meaning  superficial  plausibility — of  the 
statements  in  this  table  require  only  exegesis  to  eonvinec 
the  doubter  that,  no  matter  how  wrong  I  may  be  about 
his  group  (analyst  or  scientist),  I  have  fairly  anatomized 
the  group  to  which  he  does  not  belong.  I  ean  provide 
documentation  for  most  of  the  statements  in  the  scien¬ 
tist  column — the  volume  of  literature  is  large  enough  to 
prove  almost  any  point.  Contrarywise,  I  ean  document 
none  of  the  statements  in  the  intelligence  analyst  col¬ 
umn,  and  welcome  any  documented  disproof. 

Use  of  oral/ informal  information 

This  would  seem  to  be  the  scientist’s  favorite  mode 
of  communication — in  his  own  corridors,  by  telephone, 
or  at  meetings  held  in  pleasant,  distant  and  preferably 
overseas,  locations.  It  is  quite  clear  from  the  literature 
that  corridor  gossip  is  the  best  part  of  such  meetings — 
that  papers  are  attended  only  in  desperation  or  in¬ 
clement  weather. 

It  is  my  impression  that  in  intelligence  processing 
such  oral/informal  communications  are  reduecd  to 
writing  at  the  earliest  possible  stage,  and  that  the  strict 
compartmentalization  mandatory  because  of  security 


tends  to  diseourage  casual  subjeet-oriented  conversa¬ 
tion  in-house. 

Use  of  library  and  place  of  reading 

The  frequency  of  a  user’s  contact  with  his  library/ 
information  center  tends  to  be  inversely  proportional  to 
his  distance  from  it.  The  curve  is  not,  however,  mono- 
tonie — as  those  who  have  listened  to  arguments  about, 
say,  departmental  libraries  versus  centralized  libraries 
know,  there  seems  to  be  a  forbidden  zone,  with  the 
lower  edge  dimensioned  in  hundreds,  or  at  best  thou¬ 
sands,  of  feet  within  which  a  scientist  will  not  travel 
to  get  information  from  a  library.  The  upper  edge,  at 
least  in  days  of  grantsmanship  and  easy  travel  funds, 
is  probably  in  the  order  of  thousands  of  miles.  The 
intelligence  analyst  with  a  library  on  the  premises  has 
a  considerable  advantage  over  the  scientist,  on,  say,  a 
eampus  like  UCLA  where  the  library  is  too  far  to  walk 
and  parking  is  almost  impossible  if  he  drives.  It  might 
even  be  possible  to  distinguish  between  information- 
oriented  and  lab-oriented  scientists  by  releasing  suitably 
marked  specimens  at  varying  distances  from  a  library. 

A  good  university  library  ean,  cither  from  its  own 
holdings  or  through  the  informal  but  highly  effective 
library  community,  provide  the  user  seeking  an  iden- 
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tificd  item  with  a  copy,  a  reasonably  legible  surrogate 
or,  in  the  ease  of  incunabula  and  similar  rarities, 
identification  of  the  physical  location  of  almost  every 
book  and  scientific  journal  ever  published.  It  can  also 
provide  a  descriptive  catalog  of  every  bibliographic 
item  that  should  be  on  its  shelves.  But  its  subject 
cataloging  of  its  own  holdings,  let  alone  those  of  other 
libraries,  lags  far  behind.  (Perhaps  because  books  are 
more  fun  to  buy  and  hold  than  catalogs,  and  especially 
catalogers.)  In  effect,  the  bargain  that  a  library  makes 
with  its  more  sophisticated  users  is,  “If  you  can  tell  me 
what  you  want,  I'll  get  it  for  you  (if  you’re  prepared 
to  wait)  but  it’s  up  to  you  to  find  out  what  you  want.” 

The  precise  converse  of  this  situation  may  occur  in 
intelligence  installations  which  have  concentrated  on 
mechanizing  their  bibliographic  reference  system,  per¬ 
haps  even  to  the  point  of  providing  abstracts  of  docu¬ 
ments  of  interest,  without  making  equally  good  provi¬ 
sions  for  providing  full-text  copies  of  the  documents 
on  which  the  bibliographic  apparatus  is  presumably 
based.  Analysts  become  very  unhappy  when  the  com¬ 
puter  tells  them  about  documents  they  can’t  get;  many 
analysts  say  that  90  per  cent  of  their  time  is  spent  in 
getting  their  hands  on  the  document  they  want  after 
they  have  found  out  they  want  it.*  Perhaps  the  implicit 
promise  of  delivery  should  be  made  more  explicit. 

Or,  if  the  traditional  library  resembles  the  traditional 
Hollywood  librarian,  hiding  all  sorts  of  interesting  physi¬ 
cal  resources  behind  a  dowdy  facade  till  the  last  reel, 
the  unwisely  mechanized  information  system  may  re¬ 
semble  a  fan-dancer — all  index  and  no  delivery. 

Maintain  personal  files 

Jahoda  (Jahoda,  1966)  in  work  done  for  AFOSR 
has  found  that  46  of  a  sample  of  75  research  workers 
maintained  personal  files.  This  seems  to  be  in  accord 
with  values  reported  in  the  literature  of  from  45  to 
100  per  cent.  Wallace  (Wallace,  1964)  has  described 
the  preparation  of  personal  indexes,  printed  with  the 
aid  of  computers  for  professional  and  administrative 
personnel  at  Systems  Development  Corporation.  To  the 
best  of  my  nowiedge,  most  discussions  of  mechaniza¬ 
tion  of  intelligence  information  processing  have  con¬ 
centrated  on  mass  processing  at  the  central  facility.  It 
would  seem  entirely  possible  that  perhaps  more  com¬ 
puter  support  could  be  given  in  maintenance  of  indi¬ 
vidual  analysts’  files.  There  must  be  something  better 
than  the  present  untidy  hoards. 

*By  way  of  contrasl,  a  worker  in  a  fairly  specialized  informa¬ 
tion  center  in  the  nuclear  field  tells  me  that  his  time  is  spent 


as  follows: 

Finding  and  retrieving  references  10-15% 

Finding  and  retrieving  documents  10% 

Retrieving  data  from  the  above  sources  50% 

Comparison,  evaluation  and  report  writing  30% 


Good  Housekeeping  Seal  of  Approval 

Pasternack  (Pasternack,  1966)  editor  of  The  Physi¬ 
cal  Review ,  is  the  latest  journal  Brahmin  to  point  out 
that  the  primary  purpose  of  the  journal  is  to,  and  I 
am  paraphrasing  wickedly,  substitute  the  value  judg¬ 
ments  of  the  referees  and  editor  for  that  of  the  indi¬ 
vidual  scientist.  “If  you  read  it  in  Physical  Review ,  you 
can  be  almost  sure  it’s  true — but  if  it’s  in  a  report, 
you’d  better  be  ready  to  snort.”  (Anonymous,  1966) 

The  scientist  given  a  paper  to  evaluate  would  be 
quite  unhappy  if  all  descriptive  information  were  de¬ 
leted.  Before  he  ever  reads  it  he  wants  to  know  who 
wrote  it,  where  he  worked  (both  laboratory  and  coun¬ 
try),  where  and  when  it  was  published,  perhaps  even 
what  agency  supported  the  work.  There  arc  certain 
internal  tests  he  can  and  does  apply:  Arc  the  curves 
given  without  any  experimental  points  or,  even  worse, 
do  all  the  points  fall  precisely  on  the  curves?  Does  the 
author  recognize  and  explain  any  inconsistencies?  Are 
there  a  reasonable  number  of  references,  and  is  the 
purpose  of  including  each  reference  clear,  or  arc  they 
just  the  window’  dressing  of  a  pseudo-erudition?  (Per¬ 
haps  the  same  pseudo-erudition  which  drives  me  to 
mention  Gocdel’s  theorem  on  formally  undecidable 
propositions — that  it  is  impossible  to  demonstrate  the 
non-contradictoriness  of  complex  systems  without  going 
outside  those  systems.) 

Apparently  there  arc  times  when  intelligence  finds 
it  necessary  to  deprive  the  analysts  of  these  useful 
clues,  and  interpose  a  middleman  between  the  col¬ 
lectors  and  the  analyst: 

“The  middleman  grades  the  data  for  reliability  of 
source  and  accuracy  and  reliability  of  content.  .  .  .  (He) 
according  to  standard  practice,  is  restricted  to  a  very 
narrow  language  in  making  his  evaluations.  He  is  per¬ 
mitted  to  grade  the  reliability  of  the  source  according 
to  the  letters  A,  B,  C,  D,  and  the  content  according  to 
the  numbers  1,  2,  3,  4.  Thus  A-l  would  designate  a 
report  of  unvarnished  truth  that  was  straight  from  the 
horse's  mouth.  ...  If  the  data  happen  to  have  come 
from  a  document,  a  newspaper  or  press  release  or  some 
such,  one  school  of  evaluators  simply  designates  their 
value  with  the  single  word  ‘documentary.’  .  .  .  Often 
middlemen  have  no  independent  line  on  the  reliability 
of  the  source,  and  instead  of  admitting  as  much,  will 
proceed  to  grade  the  source  on  the  apparent  reliability 
of  the  content.  This  movement  in  vicious  circles  is 
neither  helpful  nor  valid.”  (Kent,  1965) 

I  am  told  that,  all  other  factors  being  equal  (or 
unavailable),  the  value  an  analyst  places  on  an  item 
tends  to  vary  directly  with  the  classification  of  that  item. 

Use  of  mechanized  current  awareness  services 

Descriptive  cataloging,  that  phase  of  the  process  of 
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cataloging  which  concerns  itself  with  the  identification 
and  description  of  books,  is  a  deceptively  simple  proc¬ 
ess.  Most  non-library  oriented  beginners  in  the  field  of 
information  tend  to  discover  descriptive  cataloging  like 
Napoleon  discovered  Russia — the  first  steppes  are  easy, 
but  it  gets  tougher  the  further  you  go.  The  title  of  a 
publication  is  part  of  its  descriptive  cataloging.  If  this 
title  is  key-punched,  it  can  be  manipulated  in  various 
ways  to  give  indexes  known  as  KVVIC  (Key  Word  In 
Context),  KWAC  (Key  Word  And  Context)  and 
KWOC  (Key  Word  Out  (of)  Context). 

A  review  by  Stevens  (Stevens,  1965)  found  more 
than  40  examples  of  KW1C  and  its  congeners  as  of 
February  1964.  Perhaps  8  of  these,  especially  those 
produced  by  Chemical  Abstracts  Service,  Biological 
Abstracts  and,  of  course,  that  of  the  ACM,  have  passed 
the  test  of  the  marketplace  even  if  they  arc  not,  per¬ 
haps  “the  miracle  of  the  decade/'  (Baker,  1960) 

As  the  co-designer  of  one  such  system,  WADEX 
( Rippcrbcrger,  Wooster  and  Juhasz,  1964),  I  am  keen¬ 
ly  aware  of  its  limitations — an  almost  inescapable 
bulkiness,  caused  by  the  necessity  of  replicating  titles 
as  many  times  as  there  arc  significant  words,  and  the 
miserable  inadequacies  of  titles  as  written  by  the 
average  author.  Later  forms  of  WADEX,  especially 
WADEX  III,  and  presumably  its  rivals,  have  learned 
to  handle  enriched  titles — primitive  subject  terms 
added  to  the  author’s  original  title — and  class  num¬ 
bers.  I  would  imagine  that  any  installation,  intelligence 
or  no,  that  key-strokes  descriptive  cataloging  informa¬ 
tion  and  adds  a  classification  number  might  find  sonic 
form  of  KWIC  index,  sorted  by  class  numbers,  a  cheap 
and  not  too  nasty  method  of  setting  up  a  current  aware¬ 
ness  system. 

A  full  Selective  Dissemination  of  Information  sys¬ 
tem  is  something  else  again  Luhn's  original  concept 
(Luhn,  1958,  1961)  assumed  that  both  documents 
and  users  would  be  indexed  by  some  form  of  coordinate 
indexing,  and  that  the  abstracts  chosen  by  machine 
matching  of  terms  would  be  distributed  to  individual, 
named  users.  It  is  my  impression  that  there  is  a  grow¬ 
ing  tendency  to  drop  this  personalized  service  and 
substitute  the  distribution  of  sets  of  abstracts  to  classes 
of  users.  Barrett,  for  example  (Barrett,  1965),  in  de¬ 
scribing  CIRC,  says  that:  “A  user  profile  is  a  list  of 
topic  tags,  or  descriptors,  which  describe  the  scope  of  a 
user  group’s  interest.  Note  that  I  said  group.  All  of 
our  dissemination  is  based  upon  unit  profiles.  We  dis¬ 
covered  early  on  that  individual  profiles  had  a  very 
high  degree  of  duplication,  and  that  it  appeared  more 
economical  to  talk  in  terms  of  a  profile  serving  a  unit 
rather  than  an  individual.  Such  a  unit  might  have  two, 
three,  five,  or  even  10  people  in  it  working  on  closely 
associated  subject  areas.” 


I  see  no  reason  why  a  perfectly  good  SDI  system 
could  not  be  made  to  work  based  completely  on  an 
hierarchical  subject  classification  system,  assigning 
classification  numbers  to  documents  and  to  user  groups. 

Ambience — feed-back  on  products 

Perhaps  the  principal  difference  between  the  intelli¬ 
gence  analyst  and  the  scientist  lies  in  their  ambience. 
A  scientist  lives  in  a  highly  resonant  environment  He 
talks  to  his  peers  informally,  he  presents  papers  at 
meetings,  he  publishes  papers  and  receives  reprint  re¬ 
quests,  people  write  him  letters  of  praise  or  otherwise. 
If  he  is  as  much  of  a  sehnook  as  most  of  us  are,  he 
judges  other  people’s  papers  and  bibliographies  by 
whether  or  not  they  have  cited  his  own  papers.  (Ad¬ 
vanced  eases  read  other’s  papers  as  they  read  the  news¬ 
papers,  only  instead  of  turning  to  the  cornies  they  turn 
to  the  bibliography.)  Certainly  in  the  village  community 
that  science  was,  if  not  in  the  concatenation  of  conurba¬ 
tions  that  it  has  become,  the  scientist  continually  re¬ 
ceived  feed-back  on  his  work. 

Not  so  the  analyst.  He  finishes  writing  a  report,  turns 
it  in  to  his  supervisor,  has  it  bounced  once  or  twice  on 
general  principles,  OKs  the  final  copy  and  turns  it 
loose.  And  that’s  that.  Period,  paragraph  Time  to 
start  a  new  report.  If  he  has  been  allowed  to  retain  a 
strong  prose  style  and  a  gift  for  felicitous  phrasing,  he 
may  recognize  sentences  or  whole  paragraphs  of  his 
own  in  reports  prepared  by  others,  but  will  search  in 
vain  for  quotation  marks  or  proper  bibliographic  cita¬ 
tion.  He  knows  he  wrote  it,  but  so  what? 

Once  upon  a  time  the  old  Air  Research  and  De¬ 
velopment  Command  had  a  problem  like  this.  Scientists 
and  engineers  working  on  classified  projects  envied 
their  unclassified  colleagues  who  got  to  present  papers 
at  meetings*.  ARDC’s  solution  was  the  invention  of  the 
ARDC  Science  Seminar.  Held  under  proper  security 
safeguards,  it  provided  a  classified  forum  for  classified 
papers,  with  perhaps  a  trace,  but  only  a  trace,  of  Air 
Force  hoop-la  in  panels  of  judges,  and  awards  for  the 
best  papers  and  best  presentations. 

I'm  not  quite  sure  that  it  takes  a  Herman  Kahn  to 
think  of  the  unthinkable,  and  visualize  intra-  and  per¬ 
haps  even  (shudder)  inter-agency  competitions  for  the 
best  intelligence  reports.  We  had  amazing  luck  with 
a  similar  problem  in  our  workshop  on  “Working  with 
Semi-Automatic  Document  System”  by  inviting  only 
those  people  who  never  gave  papers  at  meetings,  but 
stayed  home  and  did  the  work  while  their  bosses  took 
the  credit.  To  the  best  of  our  ability  it  was  an  all- 
Indian  conference.  (Those  chiefs  who  muscled  their 
way  in  usually  wound  up  out  of  harm’s  way  in  a  session 
on  system  design  and  evaluation.) 

I  understand  that  there  arc  two  classified  scientific- 
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technical  journals  in  the  intelligence  field  with  the  usual 
editorial  paraphernalia  and  that  one  of  them  in  fact 
does  award  yearly  prizes  for  the  best  papers.  This  is 
fine  as  far  as  it  goes,  but  publication  without  reader 
feed-back  (the  average  author  of  the  average  article  in 
a  scientific  journal  usually  receives  10-15  reprint  re¬ 
quests;  that  same  article  is  unlikely  to  be  read  by  more 
than  7  per  cent)  tends  to  be  a  fairly  harmless  form  of 
solitary  vice.  In  presenting  scientific  results,  as  in  sev¬ 
eral  other  matters,  there  really  is  no  substitute  for  a 
live  audience. 

Output  processing  methods 

As  I  have  pointed  out  elsewhere  (Wooster,  1965b, 
c)  the  basic  problem  in  technical  writing  is  to  put  it  off 
till  the  last  possible  minute,  meanwhile  building  up  vast 
untidy  heaps  of  reference  material  and  the  requisite 
nervous  tension,  and  then  converting  sources  and  ten¬ 
sion  into  finished  manuscript  as  quickly  as  possible. 

Whatever  the  potentialities  of  reactive  typewriters, 
little  is  actually  being  done  to  ease  this  private  ordeal. 
I  tend  to  regard  those  who  do  not  know  how  to  use  a 
typewriter  as  hopeless.  I  am  sure  that  anyone  who 
really  wanted  to  curb  the  “information  explosion” 
could  cut  the  production  of  papers  in  half  by  outlawing 
lined  pads,  ball-point  pens  and  paleographic  secretaries. 
I  am  indebted  to  Project  INTREX  (Overhagc,  1965) 
for  pointing  out  that  the  average  scientist  does  not 
know  how  to  type  well  enough  to  use  a  computer 
console  adequately  for  information  retrieval.  I  am  not 
overly  fond  of  solution  which  presupposes  dictating — 
such  as  the  long  anticipated  voice-operated  typewriter. 
If  cacoethes  scribendi  is  bad,  cacoethes  loquendi  is  10 
times  worse. 

Given  a  candidate  who  is  willing  to  learn  how  to 
type,  though,  it  should  not  be  impossible  to  devise  an 
inexpensive  writer’s  work  station,  complete  with  key¬ 
board,  display  screen,  and  my  own  invention,  the 
“plagiarist’s  pencil,”  which  hangs  alongside  the  light 
pen  but  is  actually  a  print  reader,  with  the  capability 
of  tracing  passages  in  text  and  transferring  them  to  my 
manuscript. 

And  while  you’re  at  it,  be  sure  to  put  wheels  on  it. 
I  want  something  that  can  be  wheeled  into  my  cubby¬ 
hole  when  I  need  it,  and  rolled  back  to  the  storeroom 
when  I  don’t.  And  the  last  thing  in  the  world  I  need 
is  a  large,  expensive,  monocular  conscience  glaring  at 
me  when  I’m  not  ready  to  start  writing.  And,  since 
creation  should  be  at  least  as  solitary  as  procreation,  the 
last  thing  in  the  world  I  want  to  do  is  to  load  my  book¬ 
shelves  onto  a  wheelbarrow  and  trundle  them  off  to 
a  brightly-lit  central  facility  with  one-way  glass  windows 
to  show  me  off  to  visitors  whenever  I  write  a  paper. 


SUMMARY 

Both  the  scientist  and  the  intelligence  analyst  are  con¬ 
cerned  with  converting  the  information  from  scientific 
and  technical  documents  into  written  manuscripts.  Al¬ 
though  the  scientist  devotes  a  large  percentage  of  his 
time  to  information  processing  this  is  only  incidental 
to  his  overt  goal,  gaining  new  knowledge  and  insight, 
and  his  covert  goal,  enhancing  his  stature  in  the  scien¬ 
tific  community  by  peer  group  recognition.  The  scien¬ 
tist,  qua  information  processor,  is  an  untrained,  part- 
time  amateur.  The  intelligence  analyst  is  a  trained, 
full-time  professional.  The  scientist  deals  largely  with 
unwritten  informal  information  sources;  the  analyst  is 
usually  confined  to  formal,  written  sources.  The  ana¬ 
lyst  labors  under  certain  handicaps  which  arc  ines¬ 
capable  concomitants  of  the  information  with  which 
he  deals  and  the  uses  to  which  it  is  put.  Mechanization 
can  provide  a  partial,  but  only  a  partial,  amelioration 
of  some  of  these  handicaps. 
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INTRODUCTION 

In  the  field  of  computing,  the  word  “input”  is  used  am¬ 
biguously.  It  can  mean  the  work  done  by  keypunch 
operators  and  others  who  prepare  programs  and  data 
for  input  to  a  computing  system,  but  it  can  also  mean 
transfer  of  information  from  cards  or  magnetic  tape 
into  a  storage  device.  In  the  latter  case,  input  is  part 
of  a  cyclic  process  that  can  go  on  indefinitely  with  little 
or  no  human  intervention;  processing  yields  results 
that  arc  stored  outside  the  computer  temporarily,  then 
brought  back.  “Acquisition”  is  our  term  for  conver¬ 
sion  of  information  from  a  form  designed  for  human 
consumption  to  one  designed  for  machine  input.  It 
is  a  difficult  area  of  automatic  language  processing, 
and  the  difficulties  experienced  with  acquisition 
schemes  are  sometimes  resolved  at  the  cost  of  making 
problems  for  those  who  would  archive  or  interchange 
files  of  text.  (Each  element  of  information  in  a  bibli¬ 
ography,  a  subject  heading  list,  etc.,  is  a  small  unit  of 
text.) 

Keyboards  and  conventions 

Given  a  block  of  text  in  the  form  of  typescript  or 
print,  acquiring  it  for  automatic  processing  requires 
some  device.  The  two  main  classes  of  acquisition  de¬ 
vices  are  those  actuated  by  keyboards  and  those  that 
automatically  decode  the  signal  from  a  photoelectric 
scanner.  Let  us  examine  some  problems  encountered 
in  working  with  keyboard  actuated  devices;  with  a 
scanner,  they  arise  in  more  tedious  versions. 

If  the  designer  of  an  acquisition  system  could  begin 
by  writing  specifications  for  the  device  he  wanted, 
without  regard  to  cost,  most  of  the  difficulties  inherent 
in  the  process  would  disappear.  To  begin  with,  he 
would  specify  production  of  paper  copy,  to  be  used  for 
proofreading,  simultaneous  with  production  of  tape  or 

‘Any  views  expressed  in  this  paper  are  those  of  the  author.  They 
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some  other  recording  for  computer  input.  He  would 
specify  a  large  set  of  character  shapes,  each  shape  to 
be  avilable  in  several  sizes,  weights,  and  so  on.  He 
would  specify  that  the  machine  operate  line  by  line, 
ing  and  superscripting  would  be  provided,  and  for  the 
construction  of  complex  formulas  the  operator  would 
ator  could  produce  when  necessary.  Thus,  subscript¬ 
ing  and  superscripting  would  be  provided,  and  for  the 
construction  of  complex  formulas  the  operator  would 
be  able  to  place  characters  anywhere  on  the  page. 

No  device  with  all  these  characteristics  is  on  the 
market  now,  although  some  of  them  have  been  ob¬ 
tained  by  modification  of  standard  devices.  Sam- 
met  (1966)  mentions  several  devices  modified  to  per¬ 
mit  easier  input  of  mathematical  formulas.  (Martin, 
1965;  Minshey,  1963;  Davis  and  Ellis,  1963).  Urdang 
(1966)  presents  a  keyboard  adopted  for  typing  dic¬ 
tionary  copy;  the  uppercase  letters  are  replaced  with 
special  characters,  so  that  capitals  must  be  indicated 
by  typing  in  red.  With  two  cases  and  two  colors,  this 
device  has  four  characters  per  key,  but  the  proof¬ 
reader  must  remember  that  a  red  per  cent  mark  is  a 
lowercase  Greek  sigma.  Kuney,  Lazorchak,  and 
Walcavich  (1966  a,  b)  also  modified  their  input  de¬ 
vices  in  some  respects.  Still,  it  is  hard  to  imagine  a  way 
of  obtaining  the  variety  of  the  printed  page  with  a 
device  that  produces  paper  copy  by  means  of  engraved 
characters  and  mechanical  action.  If  the  two  uses  of 
paper  copy —  immediate  feedback  to  the  typist,  and 
subsequent  proofreading  — are  separated,  the  first  can 
be  served  by  a  cathode  ray  tube  display,  on  which 
more  varied  characters  and  arrangements  can  be 
created,  and  the  second  can  be  served  by  proof  copy 
generated  somewhere  other  than  the  typist’s  device. 

Even  if  the  designer  could  specify  such  a  device,  he 
would  have  to  specify  the  training  of  his  typists  in  ad¬ 
dition.  As  the  keyboard  and  its  ancillary  controls  be¬ 
come  more  complex,  the  training  problem  increases, 
and  brings  in  two  new  difficulties.  If  the  necessary 
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training  is  too  rigorous,  many  candidates  will  be  un¬ 
able  to  master  it.  And  few,  if  any,  will  be  able  to  at¬ 
tain  the  keyboard  speed  of  the  ordinary  typist  working 
at  the  ordinary  keyboard. 

The  practical  designer,  therefore,  adopts  a  differ¬ 
ent  approach,  doing  the  best  he  can  with  commercially 
available  devices  (perhaps  slightly  modified)  and  the 
skills  he  can  hire  (perhaps  slightly  enriehed).  If  he 
must  deal  with  the  Cyrillic  alphabet  for  one  job,  and 
with  the  Greek  and  Roman  alphabets,  using  various 
sets  of  diacritics,  for  other  jobs,  he  adapts  his  machine 
to  each  job  instead  of  trying  to  adapt  it  to  the  whole 
flow  of  jobs  once  and  for  all.  Thus,  he  ean  purchase  a 
device  equipped  with  Cyrillie  type,  and  hire  trained 
typists  for  the  How  of  Russian  text;  he  can  buy  other 
deviees,  and  hire  other  typists,  for  different  flows.  If 
he  purchases  a  device  with  interchangeable  type,  then 
only  a  part  of  the  deviee  has  to  be  replaced  in  going 
from  one  job  to  another.  If  his  machine  does  not  pro¬ 
duce  paper  eopy,  changing  from  Roman  to  Cyrillie 
is  only  a  matter  of  changing  conventions  — no  hard¬ 
ware  alterations  are  made,  but  the  interpretations  of 
the  encoded  characters  transmitted  to  the  computer 
are  altered. 

One  set  of  conventions  is  not  even  appropriate 
to  the  full  range  of  jobs  to  be  done  in  one  alphabet. 
Thus,  for  example,  both  French  and  English  are  typed 
in  the  Roman  alphabet,  but  with  a  limited  character 
set  it  would  be  unwise  to  establish  the  Freneh  dia¬ 
critics  on  the  keyboard  on  a  permanent  basis;  rather, 
by  convention,  they  should  oceupy  eertain  positions 
when  Freneh  is  typed,  but  those  same  positions  should 
be  used  for  other  characters  when  English  is  in  pro¬ 
cess.  If  a  given  manuscript  contains  speeial  marks, 
such  as  those  used  in  poetic  scansion,  conventions 
must  be  adopted  for  that  one  manuscript;  it  would  be 
most  foolish  indeed  to  keep  those  marks  on  the  key¬ 
board  after  the  one  manuscript  had  been  completed. 
If  some  of  the  text  to  be  treated  contains  bold  face, 
a  conventional  way  of  indicating  a  bold-faee  segment 
must  be  adopted,  but  the  character  used  for  that  pur¬ 
pose  ean  have  a  different  use  when  text  without  bold 
faee  is  being  acquired. 

Normalization 

If  we  now  consider  the  case  of  a  laboratory  in  whieh 
the  conventions  for  use  of  keyboard  actuated  devices 
vary  as  freely  as  this,  we  can  see  that  archiving  will 
pose  grave  difficulties.  In  a  sense,  eaeh  batch  of  text 
has  been  acquired  in  a  distinctive  encoding  scheme. 
There  is  not  even  a  closed  set  of  encoding  schemes, 
since  a  user  may  appear  at  any  time  who  requires  a 
set  of  conventions  different  from  any  set  used  in  the 
past.  If  a  library  of  all  text  acquired  by  the  laboratory 
is  maintained  permanently,  the  encoding  conventions 


for  each  unit  stored  must  be  retained  because  without 
them  text  is  unreadable.  All  processors  working  on 
text  must  be  constructed  in  variant  forms  for  the  dif¬ 
ferent  encoding  schemes,  or  else  adapted  by  means 
of  parameters  to  the  many  possibilities. 

The  current  situation  with  regard  to  interchange  of 
textual  materials  among  research  centers  or  among 
operating  entities  such  as  libraries  is  about  like  the 
situation  in  our  hypothetical  laboratory.  The  univer¬ 
sity  librarian  will  discover,  not  too  long  from  now,  that 
many  flows  of  bibliographic  information  are  available 
on  magnetic  tape.  Each  is  encoded  in  a  way  that  was 
considered  appropriate  to  his  purpose  by  some  de¬ 
signer.  the  university  librarian  — and  the  library’s 
programmer— may  find  that  all  the  encoding  schemes 
are  about  equally  acceptable,  but  no  one  is  sufficient. 
The  library  requires  all  of  the  tlow  of  information, 
and  for  many  purposes  and  in  many  libraries  it  may 
be  considered  necessary  to  merge  the  several  flows. 

It  should  be  self-evident  that  universal  adoption  of 
the  American  Standard  Code  for  Information  Inter¬ 
change  would  be  no  more  to  the  point  than  universal 
adoption  of  a  single  brand  and  model  of  paper-tape 
perforator.  It  is  not  at  the  level  of  the  ASCII,  or  the 
perforator  as  a  physical  device,  that  the  designer 
solves  his  major  difficulties.  He  must  deal  with  a  far 
greater  range  of  characters  than  ASCII  code,  or  a 
paper-tape  perforator,  is  designed  to  handle.  The  large 
set  of  characters  is  essential  for  satisfactory  repre¬ 
sentation  of  the  eontent  of  texts.  What  is  required  is 
a  uniform  way  of  dealing  with  indefinitely  increasing 
variety  of  characters  and  arrangements  of  characters. 

To  return  once  more  to  a  hypothetical  laboratory  in 
which  language  processing  is  performed  on  the  grand 
scale,  the  laboratory  eould  have  a  single  encoding 
scheme  encompassing  all  characters  known  to  exist 
in  text  of  interest,  and  expansible  to  include  any  char¬ 
acters  discovered  in  the  future.  The  encoding  scheme 
would  take  into  aecount  variations  of  size  and  weight, 
features  of  arrangement,  etc.  In  order  not  to  reimpose 
the  same  old  difficulties  on  the  acquisition  system, 
the  laboratory  could  have  a  program  with  which  to 
mediate  between  acquisition  encoding  and  archival 
encoding.  Various  conventions  would  be  adopted,  as 
described  in  earlier  paragraphs;  but  the  output  of 
the  keyboard  actuated  device  would  not  be  considered 
the  archival  form.  An  acquisition  program  would  work 
on  the  recording  produced  by  the  typist,  taking  into 
aeeount  all  temporary  alterations  of  hardware  and  all 
special  conventions  adopted  for  the  given  manuscript. 
This  program  would  yield  an  archival  tape  in  whieh 
every  character,  and  every  feature  of  arrangement, 
was  immediately  distinguishable  from  all  other  char¬ 
acters  and  features  of  arrangement  found  in  all  of  the 
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manuscripts  ever  acquired.  Thus,  any  program  written 
for  processing  text  in  this  laboratory  could  be  written 
in  a  single  version,  taking  the  archival  form  as  input, 
and  therefore  capable  of  operating  on  the  entire  ar¬ 
chive  with  no  special  adjustments. 

As  for  the  problem  of  interchange,  it  is  clear  that 
those  who  use  a  common  archival  format  can  ex¬ 
change  material  without  much  difficulty.  Laboratories 
that  have  an  acquisition  program  of  the  kind  suggested 
can  accept  information  in  nonstandard  encoding  by 
adjusting  the  acquisition  program  to  the  characteris¬ 
tics  of  the  outside  source.  Thus,  the  acquisition  pro¬ 
gram  can  facilitate  collection  of  material  from  the 
publishing  industry,  and  from  all  other  uncontrollable 
sources. 

The  RA  ND  acquisition  program 

To  be  of  practical  use,  the  acquisition  program 
must  be  adaptable  to  a  wide  variety  of  input  devices 
and  conventions  for  using  them,  since  the  whole  in¬ 
tention  is  to  vary  the  conventions  as  often  and  as 
much  as  necessary  to  make  the  keyboard  job  easy.  A 
program  of  this  kind  has  been  written  at  The  RAND 
C  orporation  (Graves,  et  al.,  1966).  It  need  not  serve 
forever,  although  it  is  in  practical  use  now;  it  should  be 
redesigned  and  rewritten  as  soon  as  ways  of  making 
it  more  convenient  become  clear. 

The  user  of  the  RAND  program  designates  certain 
characters  on  his  input  device  as  shifts.  Naturally,  the 
shifts  defined  by  the  hardware  of  the  device  in  use  are 
ordinarily  among  these.  For  each  shift,  the  user  pro¬ 
vides  a  complete  table  of  interpretations  of  the  punch- 
able  characters.  Thus,  many  changes  of  character  sets, 
by  alternation  of  hardware  or  by  change  of  convention, 
amount  to  nothing  more  than  changes  in  one  of  these 
tables. 

Sometimes  it  is  desirable  to  expand  the  character 
set  by  allowing  for  strikeovers.  Thus,  overstriking  a 
lower  case  c  with  a  slanted  line  makes  the  convention¬ 
al  symbol  for  “cents."  The  RAND  program  allows 
for  such  conventions  by  moving  a  pointer  backward  as 
well  as  forward  through  an  array  of  character  posi¬ 
tions,  and  allowing  for  combination  of  a  character 
representation  already  stored  with  a  new  one  struck 
at  the  same  position.  Of  course,  underscoring  and 
diacritics  are  readily  treated  in  this  way. 

Urdang's  keyboard  provides  overstriking  for  dia¬ 
critics  and  underscoring,  but  his  conventions  call  for 
some  words  to  be  treated  as  underscored  (and  ulti¬ 
mately  set  in  a  special  type  face)  when  the  typist 
underscores  only  the  initial  letter.  This  convention 
makes  the  character  a  shift  for  the  RAND  program; 
succeeding  characters  would  be  coded  as  in  the  ap¬ 
propriate  type  face.  This  convention  also  makes  the 


spacebar  a  shift,  putting  the  system  back  into  normal 
shift. 

In  core  storage,  the  RAND  acquisition  program 
sets  aside  a  region  as  a  page  array.  Like  a  sheet  of 
paper  being  fed  through  a  typewriter,  this  region  is 
divided  into  rows  and  columns.  In  a  machine  with  36- 
bit  cells,  one  full  cell  is  reserved  for  each  character 
position.  As  each  row  of  characters  is  filled,  a  test 
can  be  performed  to  determine  whether  an  indepen 
dent  multiline  segment  of  input  — a  “page"  — has  been 
completed.  The  input  is  therefore  processed  line  by 
line  until  a  page  is  filled,  whereupon  analysis  and  out¬ 
put  processing  of  the  page  is  performed. 

In  many  input  streams,  functionally  distinct  ele¬ 
ments  are  differentiated  by  position  and  typographic 
features.  Thus,  a  chapter  title  may  be  distinguished 
by  size  and  face  of  type,  by  blank  space  above  and 
below,  by  centering  horizontally  on  the  page,  and  so 
on;  or  a  chapter  title  might  be  distinguished  rather  by 
inclusion  of  the  word  “chapter"  Hush  left  at  the  top  of 
the  page,  with  a  number  following  and  a  string  of 
underscored  characters  completing  the  line.  These 
format  features  can  be  interpreted  by  two  additional 
parts  of  the  RAND  acquisition  program.  The  format¬ 
ter  makes  straight-line  cuts  vertically  or  horizontally 
on  the  page,  dividing  the  page  into  boxes.  If  the  input 
is  simple  enough,  the  formatter  alone  can  isolate  all 
functionally  distinct  elements.  For  example,  a  bibh- 
orgaphy  format  might  consist  of  double-spaced  ele¬ 
ments  each  separated  from  those  before  and  after  by 
extra  double  spaces.  In  some  interesting  streams  of 
material,  at  least  certain  kinds  of  boxes  will  have  to 
be  analyzed  by  a  parser.  A  parser  can  use  features  of 
individual  characters,  for  example  whether  a  char¬ 
acter  position  is  blank,  contains  a  letter  or  figure, 
contains  underscoring  or  certain  punctuation  marks, 
and  isolate  — for  example —  author,  title,  and  place  of 
publication  in  a  citation.  Our  parser  produces  a  de¬ 
scription  of  a  specific  string  of  text  by  referring  to  a 
grammar.  The  identity  of  a  subsegment  is  determined 
only  when  identification  of  the  subsegment  will  lead 
to  a  proper  interpretation  of  the  structure  of  the 
entire  stretch 

After  the  formatter  has  established  box  outlines  and 
the  parser  has  analyzed  whatever  boxes  have  complex 
internal  structure,  a  selector  brings  out  elements 
with  specified  function  and  a  reeoder  converts  the 
internal  representation  into  that  required  for  output. 
Output  may  be  to  a  printer,  another  processor,  or  an 
archiving  program  that  writes  tapes  for  permanent 
storage. 

The  usefulness  of  programs  like  the  formatter  and 
parser  has  been  questioned,  for  example  by  Buck- 
land  (1963  b,  1965).  The  information  provided  by 
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punctuation,  type  face,  and  arrangement  is  sometimes 
ambiguous  and  often  difficult  to  disentangle;  if  the 
original  typist  puts  in  explicit  signals  to  identify  ele¬ 
ments  of  the  text,  the  decoding  job  is  easier.  For¬ 
matting  becomes  an  output  problem  For  example, 
Kuney,  Lazorchak,  and  Walcavich  (1966  a,  b),  use 
code  symbols  like  title;  these  symbols  are  typed  in 
red,  and  interpreted  as  macros  calling  for  certain  type 
faces  and  sizes  and  certain  positioning.  Functional 
coding  has  the  advantages  propounded  for  it,  but  if  a 
large  corpus  in  which  functional  coding  was  not  used 
is  offered,  the  formatter  and  parser  simplify  the  work 
of  making  it  usable.  Furthermore,  once  functional 
symbols  replace  ordinary  typography  in  the  typing  of 
drafts,  the  copy  must  be  put  through  a  computer  be¬ 
fore  it  can  be  distributed;  in  some  operations,  the 
original  material  must  be  presentable,  even  though  a 
machine-readable  version  is  to  be  retained  for  some 
secondary  purpose. 

If  a  stream  of  input  does  contain  functional  encod¬ 
ing,  it  can  be  handled  in  one  of  several  ways.  The 
RAND  acquisition  program  could  be  modified  to  ac¬ 
cept  macros  that  would  store  a  title,  for  example,  not 
as  punched  in  the  input  tape  but  as  wanted  in  the  out¬ 
put.  Although  this  would  be  possible,  it  seems  ex¬ 
actly  contrary  to  the  spirit  of  functional  encoding. 
The  other  two  possibilities  begin  by  treating  function¬ 
al  encoding  as  an  extra  set  of  characters.  A  user  of  the 
RAND  program  often  allows  one  bit  to  signify  that 
the  character  is  a  letter,  another  to  identify  it  as  a 
numeric  digit;  he  can  use  a  different  bit  to  mark  a 
character  as  a  macro  symbol.  Then,  in  writing  an  ar¬ 
chival  version  of  the  text  on  magnetic  tape,  the  user’s 
program  can  put  the  functional  encoding  either  in 
the  text  stream  itself,  or  in  a  label  record  associated 
with  the  text. 

Each  user  of  the  RAND  acquisition  program  must 
describe  his  input  device  and  his  conventions  for  using 
it  in  detail.  The  acquisition  program  that  exists  today 
provides  several  formats  for  writing  different  parts 
of  the  description.  To  some  extent,  the  formats  pro¬ 
vided  are  different  because  the  program  allows  more 
complexity  at  some  points  than  at  others.  Other  dif¬ 
ferences  are  not  so  easy  to  explain  in  rational  terms. 

The  program  which  develops  line  arrays  in  storage 
by  reading  input  characters  and  consulting  tables  is, 
conceptually,  a  finite-state  machine.  Its  state  at  any 
moment  can  be  given  by  naming  a  shift  and  a  char¬ 
acter  position.  It  will  put  an  image  of  the  next  input 
character  in  that  position,  choosing  the  image  by  con¬ 
sulting  the  table  for  that  shift.  The  user  specifies,  for 
each  character,  its  image  in  each  shift  and  the  state 
to  which  it  tells  the  machine  to  pass.  Any  input 
character  can  cause  the  machine  to  go  next  to  any 


shift,  but  position  movements  are  of  a  few  kinds  only. 
No  change,  one  space  forward,  one  space  back,  to 
the  next  tab  stop,  to  the  left  margin,  and  to  the  next 
line  are  the  only  ones  used  at  present.  In  view  of  the 
power  of  subsequent  processors,  an  application  call¬ 
ing  for  greater  power  at  this  stage  would  be  hard  to 
imagine. 

The  program  for  testing  ends  of  pages  allows  the 
user  to  write  regular  expressions  over  line  descrip¬ 
tions.  Each  line  description  in  turn  is  written  as  a 
regular  expression  over  characters.  When  a  line  is 
finished,  the  test  routine  examines  the  last  line  con¬ 
structed  and  the  first  line  description.  If  the  line 
satisfies  the  description,  the  routine  moves  to  the 
preceding  line  and  the  following  description. 

The  lowest  level  reached  in  the  description  of  an  in¬ 
put  device  is  below  that  of  single  characters.  Each 
character  position  in  storage  occupies  a  full  cell  in 
which  each  single  bit  position  can  be  used  for  a  signifi- 
can  feature,  or  a  combination  of  bit  positions  can  be 
used  for  multivalued  features.  The  lowest  level  of  de¬ 
scription  is  that  of  the  features.  On  the  next  level,  char¬ 
acters  are  described  by  reference  to  the  features  pres¬ 
ent  or  absent  in  them.  The  user  may  need  to  refer  to  a 
class  of  characters  defined  by  shared  features,  or  to  an 
individual  character  for  which  he  specifies  just  the  fea¬ 
tures  needed  to  obtain  a  unique  description.  Next,  the 
user  can  refer  to  a  string  of  characters,  or  a  class  of 
strings,  and  the  strings  can  be  placed  arbitrarily  on  the 
page,  or  can  be  specified  as  a  row  of  characters,  ex¬ 
tending  from  left  margin  to  right;  a  column  of  charac¬ 
ters,  occupying  corresponding  positions  on  consecu¬ 
tive  rows  from  top  to  bottom  of  the  page,  may  have  to  be  de¬ 
scribed  also.  The  next  level  of  description  is  a  box, 
made  up  of  all  character  positions  between  limits  that 
can  be  conceived  of  as  straight,  horizontal  or  vertical  lines  on 
the  page.  Finally,  there  is  the  level  of  the  page  itself. 
Revised  description  format 

The  RAND  acquisition  system  might  serve  its 
users  better  if  a  single  format  were  available  to  them 
for  descriptions  on  all  levels.  Let  us  consider  what 
such  a  format  might  be  like.  The  major  element  will 
be  a  procedure,  consisting  of  stipulations,  grammar, 
and  command. 

One  stipulation  will  identify  the  class  of  objects 
to  be  defined  by  the  grammar:  pages,  boxes,  columns, 
strings,  or  characters.  The  user  must  also  stipulate 
the  context  in  which  his  procedure  is  to  be  applied. 
A  page  can  only  be  defined  relative  to  a  page,  but 
narrower  contexts  can  be  specified.  A  box,  row,  etc., 
can  be  defined  relative  to  another  box.  Thus,  for  ex¬ 
ample,  a  user  might  be  seeking  a  row  of  blank  char¬ 
acters,  but  only  require  that  the  row  of  blanks  extend 
across  one  box;  outside  that  box,  the  same  row  might 
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contain  nonblank  characters  within  the  limits  of  the 
page. 

The  user  must  stipulate  whether  the  object  to  be 
defined  is  taken  as  consisting  of  rows,  columns,  or 
characters.  The  user  must  stipulate  where  the  first 
element  of  his  grammar  is  to  be  applied.  For  example, 
it  might  apply  to  the  first  row  standing  in  the  page 
array,  or  to  the  last  one.  Thereafter,  progress  through 
storage  is  determined  by  the  grammar  itself. 

Finally,  the  user  must  stipulate  whether  or  not  the 
procedure  is  recursive.  I  his  stipulation  has  two  con¬ 
sequences,  which  are  described  below;  one  has  to 
do  with  allowable  forms  of  grammar,  the  other  with 
types  of  command. 

A  grammar  is  a  sequence  of  statements,  each  with 
three  parts.  The  first  is  a  description  of  some  unit  or 
class  of  units.  The  second  and  third  are  action  clauses, 
determining  respectively  what  is  done  when  the  unit 
description  does  or  does  not  apply  to  a  specified  unit 
in  storage. 

The  description  of  a  unit  can  be  (i)  the  name  of  a 
procedure,  (ii)  a  specification  of  features,  or  (iii)  an 
ordinal  number.  If  the  procedure  is  recursive,  a  state¬ 
ment  can  contain  the  name  of  another  procedure  on 
the  same  level.  In  a  nonrecursive  grammar,  a  descrip¬ 
tion  can  be  given  by  naming  a  procedure,  but  only  one 
of  lower  level.  For  example,  a  nonrecursive  procedure 
for  defining  a  class  of  boxes  can  call  another  pro¬ 
cedure  that  defines  a  class  of  rows;  only  a  recursive 
box-definition  procedure  can  call  another  procedure 
that  also  defines  a  class  of  boxes. 

If  the  description  of  a  unit  is  given  by  a  full  spec¬ 
ification  of  features,  it  uniquely  identifies  a  type. 
Thus  the  user  can  specify,  character  by  character,  the 
entire  content  of  a  complete  box,  row,  or  column.  But 
a  description  of  a  single  character  might  include  only 
one  feature,  specifying  that  whatever  character  oc¬ 
cupies  the  position  tested  must  include,  for  example, 
the  feature  “underscored.11  Other  operators  (currently 
available  only  in  the  string-parsing  segment  of  the 
RAND  acquisition  system)  would  permit  the  user  to 
describe  a  specific  concatenation  of  named  characters, 
or  any  arbitrary  string  over  a  specified  alphabet,  or 
any  of  a  given  list  of  characters,  or  any  character  not 
on  a  given  list,  or  any  string  of  characters  not  in  a 
specified  list. 

Some  of  the  stipulations  of  a  calling  procedure  can 
be  carried  forward  to  those  it  calls.  If  a  procedure 
is  stipulated  to  define  boxes  in  terms  of  rows,  any 
procedure  it  calls  must  be  a  row-definition  procedure. 
If  procedure  B  is  stipulated  to  define  a  box  relative 
to  box  A,  then  any  procedure  called  by  B  must  be 
relative  to  box  A  also.  Hence  it  should  be  unnecessary 
to  make  all  of  the  stipulations  listed  above  explicitly; 


some  of  them  should  be  filled  out  when  the  procedure 
is  called. 

The  action  clauses  in  statements  of  a  nonrecursive 
grammar  are  concerned  only  with  movement  of 
pointers.  One  pointer  is  in  the  grammar  itself  and  is 
transferred  from  statement  to  statement.  Other  point¬ 
ers  are  in  the  storage  area;  let  there  be  one  for  each 
level.  A  storage  pointer  moves  by  characters,  rows, 
or  columns.  An  action  clause  must  therefore  specify 
whether  the  storage  pointer  is  to  move  forward  or 
backward  by  one  unit  or  not  to  move  at  all;  it  also 
specifies  where  the  pointer  in  the  grammar  is  to  go, 
but  it  would  be  convenient  to  assume  in  general  that 
if  no  movement  is  stated  it  goes  always  to  the  next 
statement  in  order.  (If  the  unit  description  is  an  ordinal 
number,  the  pointer  moves  that  many  units  ahead  or 
backward  in  storage,  if  possible.) 

When  a  statement  is  executed,  if  the  unit  descrip¬ 
tion  it  gives  is  the  name  of  a  procedure  of  lower  level, 
the  statements  of  that  lower  level  grammar  are  im¬ 
mediately  executed.  Thus  a  grammar  operating  on 
rows  might  call  a  procedure  operating  on  characters; 
the  lower  level  procedure  would  take  the  row  pointer 
at  its  current  position,  and  start  its  character  pointer 
at  the  stipulated  place  in  that  row,  relative  to  any  limits 
stipulated.  The  character  pointer  moves  back  and 
forth  under  control  of  the  lower  level  procedure  un¬ 
til  it  is  determined  that  the  row  does  or  does  not  satisfy 
the  procedure;  the  lower  level  procedure  then  returns 
control  to  the  calling  procedure,  announcing  the  out¬ 
come  “true11  or  “false.11 

When  a  nonrecursive  procedure  called  by  the  user's 
program  reaches  the  end  of  its  grammar,  it  has  po¬ 
sitioned  its  own  pointer  and  reached  a  condition  of  ac¬ 
ceptance  or  rejection.  That  is,  the  grammar  is  satisfied 
with  the  pointer  in  a  certain  place  or  there  is  no  place 
in  the  array  submitted  where  the  pointer  can  be  placed 
to  satisfy  the  grammar.  In  case  of  failure,  the  pro¬ 
cedure  should  inform  the  user's  program  of  the  cir¬ 
cumstances.  In  case  of  success,  the  nonrecursive  pro¬ 
cedure  makes  a  cut,  dividing  the  unit  to  which  it  is 
applied  into  two  units  of  the  same  level.  The  user 
must  supply  labels  for  these  two  units;  the  command 
at  the  end  of  the  procedure  is  therefore  “apply  these 
labels  to  the  units  obtained.11  In  the  present  RAND 
acquisition  system,  the  user  can  specify  whether  the 
sub  unit  on  which  the  pointer  lies  belongs  with  the 
first  or  second  of  the  two  units  obtained,  or  is  to  be 
excluded  from  both. 

A  recursive  procedure  must  be  capable  of  naming 
many  subunits  simultaneously.  The  user  calls  for  ap¬ 
plication  of  a  procedure  to  an  array;  that  procedure 
is  satisfied  if  it  can  find  a  way  of  subdividing  the  ar¬ 
ray  in  such  a  manner  that  certain  other  procedures  are 
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satisfied  for  subarrays;  and  so  on.  Thus  a  recursive 
segmentation  structure  develops,  and  when  the  whole 
array  is  found  acceptable  to  the  main  procedure  all 
of  the  subarrays  are  labeled.  The  appropriate  com¬ 
mand  to  place  at  the  end  of  a  recursive  procedure  is 
therefore  “apply  permanent  labels  to  wanted  sub- 
arrays.”  In  the  current  RAND  system,  the  parser  is 
followed  by  a  selector  that  does  just  this. 

Online  operation 

If  an  acquisition  system  of  this  kind  is  to  be  oper¬ 
ated  on  line,  that  is,  with  a  direct  link  between  the 
typewriter  and  the  computer,  it  should  be  capable  of 
following  the  typist  line  by  line  or  character  by  char¬ 
acter,  accepting  corrections  of  errors  on  the  spot  and 
interrupting  to  point  out  errors  where  possible.  The 
typist  should  be  able  to  delete  an  entire  line  rather  than 
let  it  go  into  storage.  The  typist  should  also  be  able 
to  replace  one  character  with  another,  but  if  over- 
striking  is  permitted  for  the  sake  of  enlarging  the 
character  set,  the  scheme  for  correction  by  overstrik¬ 
ing  must  be  designed  so  as  not  to  interfere.  If  the  dis¬ 
play  device  used  at  the  console  is  a  cathode  ray  tube, 
and  if  format  analysis  is  performed  during  input  typ¬ 
ing,  it  would  be  possible  to  display  the  results  of  for¬ 
matting  to  the  typist  immediately  after  a  page  of  input 
is  completed.  The  boxes  formed  by  the  formatting 
program  could  be  outlined  on  the  display  tube,  so 
that  the  typist  could  make  corrections  immediately 
if  necessary.  The  on-line  station  is. exceedingly  con¬ 
venient  for  editing,  where  input  is  obtained  from  an  un¬ 
controllable  source.  Errors  in  the  input  can  be  identi¬ 
fied  if  the  result  of  acquisition  processing  is  put  on  a 
screen  in  an  appropriate  format,  and  an  editor  with  a 
suitable  cursor  can  rapidly  make  the  minor  changes 
that  are  likely  to  be  necessary  on  account  of  input 
errors  (Buckland,  1963a). 

The  complexities  of  chemical  and  mathematical 
formulas  can  be  handled  more  conveniently  with  a 
console  having  a  powerful  cursor  mechanism  than 
with  a  typewriter  (Cossum  and  Hardenbrook,  1964; 
Martin,  1965;  Minsky,  1963).  If  a  text  contains  many 
equations  or  diagrams,  the  typist-s  job  is  inordinately 
difficult.  But  let  us  imagine  a  system  with  a  ty  pewriter 
keyboard,  RAND  Tablet  (Davis  and  Ellis,  1964), 
and  cathode  ray  tube. 

For  ordinary  text,  the  ordinary  typewriter  keyboard 
is  used.  The  text  is  displayed  on  the  screen  as  it  is 
typed.  When  an  equation  is  to  be  inserted,  the  op¬ 
erator  moves  to  a  tablet  and  adjusts  the  cursor  so  as 
to  put  the  material  in  its  proper  place  relative  to  the 
previous  text.  The  typist  then  draws  in  the  symbols 
of  the  equation  or  diagram,  putting  them  in  proper 
relative  orientation.  Such  an  operation  of  course  re¬ 


quires  character  recognition  programs,  but  character 
recognition  is  apparently  somewhat  less  difficult 
when  the  cursor  can  be  followed  as  it  moves.  Further¬ 
more,  recognition  errors  can  be  corrected  immedi¬ 
ately. 

Presumably  the  computer  is  operating  programs 
for  conversion  to  archival  encoding  at  the  same  time. 
What  encoding  scheme  should  be  used  for  chemical 
formulas  and  mathematical  equations  is  not  obvious 
yet.  One  scheme  would  be  to  encode  the  graphical 
shape  of  the  formula,  using  the  standard  symbols 
for  superscripting,  subscripting,  line  return,  and  so 
on.  The  other  possibility  is  functional  encoding.  For 
this  purpose,  a  table  of  symbols  for  summation, 
integration,  and  other  operations  would  be  stored. 
With  each  would  be  given  information  about  its  argu¬ 
ments.  The  two-dimensional  display  prepared  on  the 
Tablet  would  be  parsed,  and  what  would  be  stored  — 
presumably  under  a  special  Hag  — would  be  an  ab¬ 
stract  representation,  perhaps  in  parenthesis-free 
notation.  Then,  as  appropriate  for  information  stored 
in  the  catalog  format,  programs  suitable  to  the  dis¬ 
play  devices  at  hand  reconvert  from  abstract  repre¬ 
sensation  into  one  more  familiar  to  the  reader.  Several 
proposals  have  been  made  for  functional  or  abstract 
representation  of  chemical  formulae.  One  of  them 
might  well  be  adopted  for  use  here. 

Archives  are  worth  establishing  only  insofar  as  use 
can  be  predicted  for  them.  Some  reasonable  purpose 
must  be  foreseen,  but  that  alone  is  not  enough.  The 
information  stored  will  serve  a  good  purpose  only  if 
properly  selected  and  suitably  encoded.  Encoding 
suitability  depends  on  standardization,  and  complete¬ 
ness.  This  paper  has  discussed  problems  of  a  low  in¬ 
tellectual  order,  but  they  must  be  dealt  with.  A  com¬ 
mon  encoding  and  formatting  scheme  is  required, 
and  it  must  be  rich  enough  so  that  extracts  from  an 
archive  can  be  published  in  a  version  of  very  high 
quality  when  necessary. 
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Utilization  of  on-line  interactive  displays 
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San  I  a  Monica,  California 


INTRODUCTION 

Text  processing  on  a  computer  is  not  yet  fully  automa¬ 
tic.  The  present  state-of-the-art  in  data  and  language 
analysis  is  such  that  significant  results  can  best  be 
achieved  by  a  man-machine  system  in  which  the  com¬ 
puter  programs  are  designed  to  facilitate  on-line  inter¬ 
action  by  the  human  decision  maker. 

Three  such  interactive  programming  systems  are  de¬ 
scribed  in  this  paper.  One  is  the  General  Purpose  Dis¬ 
play  System  (GPDS),  which  provides  the  user  with  a 
convenient  tool  for  constructing  and  manipulating  a 
variety  of  visual  displays.  The  second  is  a  sentence  anal¬ 
ysis  system  (PLP  II)  that  computes  an  ddisplays  the 
syntactical  structure  of  sentences  while  the  user,  view¬ 
ing  the  results,  makes  corrections  or  selects  those  anal¬ 
yses  that  are  most  satisfactory.  The  third  is  an  inter¬ 
active  document  retrieval  system  called  BOLD  which 
helps  the  user  formulate  a  search  request  and  displays 
abstracts  of  the  retrieved  documents  on  the  scope  so 
that  lie  can  make  an  immediate  decision  regarding  their 
relevance. 

All  three  systems  operate  under  time  sharing  on  the 
AN/FSQ-32  computer  at  System  Development  Corp¬ 
oration  and  use  remote  stations. 

Equipment  configuration 

To  enhance  flexibility  and  man-machine  interaction, 
the  input  equipment  provides  a  functional  overlap  so 
that  messages  can  be  transmitted  by  more  than  one  de¬ 
vice.  The  standard  inquiry  station  consists  of  a  teletype¬ 
writer  and  a  cathode-ray-tube  display  console.  For 
GPDS,  this  equipment  is  augmented  with  an  auxiliary 
keyboard,  the  RAND  Graphic  Input  Tablet,  and  the 
control  function  selection  panel.  The  equipment  is  ar¬ 
ranged  as  illustrated  in  Figure  1. 

The  teletype 

The  basic  communication  device  is  a  standard  Model 
33  teletype.  This  teletype  is  the  only  means  of  commu¬ 
nicating  with  the  Time-Sharing  System  and  is  also  used 
to  load  the  program  into  the  0-32  computer. 


The  cathode-ray-tube  (CRT)  display  console  and 
light  pen 

The  CRT  is  the  principal  output  device.  It  displays 
both  textual  and  graphic  material  with  a  high  degree 
of  resolution,  and  has  controls  for  intensity,  focusing, 
and  display  positioning.  A  light  pen  is  attached  and  may 
be  used  as  an  other  input  device  to  inform  the  execu¬ 
tive  program  of  the  selection  of  a  display  option  or  to 
identify  some  portion  of  a  display  to  be  manipulated. 

The  auxiliary  keyboard 

The  auxiliary  keyboard  has  the  same  key  configura¬ 
tion  as  the  teletype  and  is  used  in  a  similar  fashion. 
However,  as  an  input  device  it  is  used  only  with  GPDS 
and  it  differs  from  the  teletype  in  two  main  respects: 
it  produces  no  printed  copy  and  therefore  has  no  output 
capability;  and  it  transmits  one  character  at  a  time  to 
the  scope  as  the  keys  are  depressed,  whereas,  the  tele¬ 
type  transmits  a  line  of  characters  when  the  carriage  is 
returned. 

The  RAND  Graphic  Input  Tablet 

The  RAND  Graphic  Input  Tablet  is  an  electronic  in¬ 
put  device  consisting  of  a  copper  writing  surface  and  a 
writing  stylus.  The  writing  surface  is  analogous  to  the 
display  seen  at  the  CRT  on  a  point-by-point  basis. 
Touching  the  point  of  the  stylus  lightly  to  the  writing 
surface  causes  the  analogous  point  on  the  CRT  to  be 
illuminated.  Pressing  the  point  of  the  stylus  firmly 
against  the  writing  surface  causes  the  location  of  the 
analogous  point  on  the  CRT  to  be  transmitted  to  the 
computer.  The  Graphic  Input  Tablet  can  be  used  for 
the  same  purposes  as  the  light  pen,  but  it  does  not  have 
the  limitation  of  being  able  to  respond  only  to  light 
emitted  by  the  CRT.  It  is  especially  useful  for  produc¬ 
ing  hand-drawn  displays. 

The  control  function  selection  panel 

The  control  function  selection  panel,  also  called  the 
button  box,  contains  60  pushbutton  switches,  of  which 
only  49  arc  currently  being  used.  When  a  switch  is  de- 


327 


328 


Information  System  Science  and  Technology 


RAND  GIT 


AUXILIARY 

KEYBOARD 


CRT  DISPLAY 
CONSOLE 


CONTROL 

FUNCTION 

SELECTION 

PANEL 


tTYPf 


Figure  I — The  augmented  user  station  (GPDS) 


pressed,  an  operating  command  controlling  certain  as¬ 
pects  of  the  program  is  put  into  operation. 

General  purpose  display  system * 

Of  the  three  systems  that  will  be  described,  GPDS  is, 
as  the  name  implies,  the  most  general-purpose  display 
system.  It  is  used  for  generating  and  manipulating  CRT 
displays  and  is  designed  to  be  used  by  persons  with 
varying  degrees  of  sophistication  in  data  processing  and 
computer  technology.  Built  into  the  system  is  an  ex¬ 
tensive  explanatory  text  as  well  as  error-detecting  mess¬ 
ages.  (Vorhaus,  1965;  Guillcbeaux,  1966).  The  object 
is  to  help  the  nonprogrammer  user,  such  as  a  military 

*GPDS  was  developed  by  SDC  in  performance  of  Contract 
AF  19(628 )-5 166  with  the  Electronic  Systems  Division,  Air 
Force  Systems  Command,  in  performance  of  ARPA  Order 
773  for  the  Advanced  Research  Projects  Agency  Information 
Processing  Techniques  Office,  with  partial  support  from  SDC's 
independent  research  program.  The  principal  investigators  are 
Alfred  H.  Vorhaus  and  Sally  C.  Bowman. 


commander,  a  business  manager,  or  a  scientist,  operate 
the  system. 

The  versatility  of  this  equipment  and  the  GPDS  pro¬ 
grams  can  be  illustrated  through  an  actual  example 
of  usage  by  the  salary  administration  section  at  SDC. 
To  insure  salary  consistency  for  similarly  qualified  in¬ 
dividuals  in  different  divisions  within  the  company  and 
to  insure  compatibility  of  one  company's  salary  sched¬ 
ule  with  the  industry  in  general,  salary  statistics  are  ac¬ 
cumulated  and  placed  in  a  data  bank  for  analysis  by 
GPDS. 

A  GPDS  process  has  been  written  to  analyze  a  sub¬ 
set  of  a  data  base,  calculate  the  second  order  curvilin¬ 
ear  regression  equations  for  five  specified  percentiles 
(10,  25,  50,  75,  and  90),  and  display  the  regression 
curves  in  graphical  format  on  the  scope  (Figure  2). 
This  process  also  has  the  option  of  displaying  and 
printing  a  tabic  of  the  actual  values  of  the  points  on  the 
plot  as  illustrated  in  Figure  3. 
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Figure  3 — Tabular  display  of  salary  data 


The  data  so  far  shown  have  been  for  a  single  com¬ 
pany.  Similar  data  exist  for  the  industry  as  a  whole. 
The  analyst  working  on-line  with  GPDS  may  be  inter¬ 
ested  in  comparing  the  salary  curves  of  a  single  com¬ 
pany  with  the  composite  for  the  industry  as  a  whole.  He 
can  construct  and  display  the  composite  salary  curve, 
or  build  a  more  complex  display  in  which  both  the  in¬ 
dividual  company  and  the  composite  curves  arc  pre¬ 
sented  simultaneously. 


It  is  important  to  note  that  this  particular  application 
of  GPDS  is  being  used  by  salary  administration  people, 
who,  working  with  a  user's  manual  and  some  pro¬ 
gramming  guidance,  manipulate  a  large  data  bank  and 
construct  displays  quickly  and  conveniently  to  answer 
a  variety  of  questions. 

Sentence  analysis * 

Information  retrieval  and  automatic  question-answer¬ 
ing  systems  require  a  capability  for  analyzing  statements 
and  questions  in  natural  language.  During  the  past 
years  a  number  of  automatic  sentence  parsers  have 
been  developed  but  none  provide  only,  nor  do  they  pro¬ 
vide  all,  of  the  correct  analyses.  As  a  result,  the  Lan¬ 
guage  Processing  and  Retrieval  Staff  at  SDC  has  con¬ 
centrated  its  efforts  on  building  an  interactive  system 
that  derives  a  grammar  from  manually  parsed  sentences. 
The  interactive  features  include  the  capability  for  users 
to  change  the  grammar,  to  select  one  of  several  pre¬ 
sented  parsings,  and  to  correct  errors  in  the  machine 
parsing. 

The  system  is  programmed  in  LISP  1.5  and  operates 
from  an  inquiry  station  consisting  of  a  teletypewriter 
and  a  CRT  display  unit.  The  program  system  is  called 
PLP  II,  since  it  bears  many  resemblances  to  the  Pattern 
Learning  Parser  previously  developed  and  described 
by  McConloguc  and  Simmons  (1965).  T  his  new  ver¬ 
sion,  however,  contains  several  unique  and  interesting 
features: 

•  First,  the  input  is  in  the  form  of  sentences  that 
have  already  been  dependency  analyzed.  From 
these  sentences,  the  system  derives  vocabulary 
and  grammar  rules  that  it  applies  to  new  senten¬ 
ces  of  similar  structure,  the  notion  being  that  it 
is  easier  to  develop  a  consistent  grammar  by  hav¬ 
ing  the  computer  derive  its  own  grammar  rules 
from  correctly  parsed  sentences  than  to  develop 
the  grammar  manually  by  making  a  linguistic 
analysis  of  a  large  corpus  of  English  text. 

•  As  a  second  feature,  a  dependency  analysis  and 
a  labelled  phrase  structure  tree  arc  produced  and 
the  tree  structure  displayed  for  each  sentence  that 
the  program  parses. 

•  In  addition  to  the  tree  structure,  the  program 
produces  kernel  sentences — one  for  each  sentence 
string  that  may  be  presumed  to  underlie  the  sur¬ 
face  structure  of  the  sentence  (see  Chomsky, 
1965). 

*This  research  was  sponsored  by  the  Advanced  Research  Proj¬ 
ects  Agency  Information  Processing  Techniques  Office  and 
was  monitored  by  the  Electronic  Systems  Division,  Air  Force 

Systems  Command  under  Contract  AF  19(628) -5 166  with 
SDC.  The  principal  investigator  is  Robert  F.  Simmons. 
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•  Finally,  and  most  importantly,  it  is  an  on-line  in¬ 
teractive  system.  The  users  have  the  freedom  to 
change  the  grammar,  to  correct  the  analyses  the 
system  makes,  and  to  select  from  the  several 
parsings  the  one  that  is  intuitively  best  suited  for 
his  needs. 

It  is  our  belief  that,  for  some  years  to  come,  such  a 
machine-aided  approach  will  be  most  effective  in  ob¬ 
taining  correct  analyses  of  text. 

The  following  example  will  help  explain  the  opera¬ 
tion  of  the  program 

Sentences  are  input  to  the  system  in  the  following 
fashion: 

THE  OLD  MAN  SAT  ON  THE  BEACH  Sentence 
ART  ADJ  N  V  PRFP  ART  N  Parts  of  Speech 

N  N  V  *  *V  N  *PREP  Dependencies 

This  input  is  in  the  form  of  three  strings,  where  the 
first  is  the  list  of  English  words  in  the  sentence,  the 
second  is  the  corresponding  list  of  their  parts  of  speech 
or  word  classes,  and  the  third  is  the  list  showing  the 
word  class  on  which  each  word  in  the  sentence  is  de¬ 
pendent.  Using  the  information  contained  in  these  three 
strings,  the  system  augments  its  existing  dictionary  with 
the  new  vocabulary,  word-class  items,  and  dependency 
rules.  A  dictionary  entry  is  constructed  for  each  word 
in  the  form  of  a  set  of  4-tuplcs  containing  ( 1 )  the  word 
class  for  the  preceding  word,  (2)  the  word  class  of  the 
word  itself,  (3)  the  word  class  of  the  following  word, 
and  (4)  the  word  class  on  which  it  is  dependent.  For 
the  word  MAN  in  the  previous  example,  the  dictionary 
entry  would  be: 

MAN:  ADJ-N-V,  V 

After  many  such  sentences  and  their  analyses  have 
been  input  to  the  system  and  many  such  dictionary 
entries  have  been  stored,  the  system  can  attempt  to 
parse  sentences  that  have  not  been  analyzed  previously. 
For  example,  the  following  sentence  was  analyzed  auto¬ 
matically: 

THE  BOOK  THAT  YOU  READ  IS  ON  THE  TABLE 
IN  THE  HALL. 

PLP  II  looks  up  each  word  in  its  dictionary  and 
obtains  for  each  the  set  of  4-tuplc  frames  that  it  has 
thus  far  accumulated.  Generally,  this  set  consists  of  3 
to  10  such  frames  for  each  word.  Using  the  informa¬ 
tion  provided  by  preceding  and  following  word  classes, 
the  system  is  able  to  discard  most  of  the  frames  as  being 
inconsistent  with  the  present  context.  It  is  also  able  to 
use  context  clues  within  the  sentence  to  calculate  word 
classes  for  words  that  were  not  in  the  dictionary.  It 
docs  this  by  predicting,  from  the  word-class  contexts 
of  the  preceding  and  following  word,  the  class  of  the 
word  in  question.  (A  detailed  description  of  the  opera¬ 
tion  of  the  system  is  available  in  Burger  et  al. ,  1966.) 


The  result  of  this  phase  of  the  program  is  a  depend¬ 
ency  analysis  of  the  sentence.  By  means  of  a  display, 
the  user  may  examine  each  string  in  the  analysis  and 
correct  any  errors.  He  may  also  request  a  display  of 
the  phrase-structure  tree  for  the  sentence  (Figure  4). 
This  tree  is  automatically  constructed  from  the  depend¬ 
ency  analysis  information  with  the  aid  of  a  brief 
phrase-structure  grammar.  As  is  always  the  case,  addi- 


Figure  A — A  phrase  structure  analysis 


tions,  deletions,  and  modifications  can  be  made  on-line. 

Although  the  PLP  II  system  is  still  in  an  early  stage 
of  development,  it  has  proven  to  be  a  valuable  research 
tool.  It  is  particularly  useful  because  the  researcher  can 
interact  with  the  parsing  system  in  an  on-line  mode.  He 
can  select  one  of  the  parsings  offered;  he  can  correct 
errors;  he  can  augment  the  grammar;  and  he  can 
modify  sentences  to  gain  new  insights  into  the  grammar 
of  the  language.  Furthermore,  he  can  perform  all  of 
these  operations  rapidly,  while  his  interest  with  the 
problem  is  current. 

BOLD  ( Bibliographic  On-Line  Display)* 

The  third  interactive  data-base  processing  program 
is  a  document  storage  and  retrieval  system  called 
BOLD  (Borko  and  Burnaugh,  1966).  The  system  was 
designed  to  allow  the  user  to  search  for  information  in 
a  file  of  magnetically  coded  and  stored  document  ab¬ 
stracts  in  much  the  same  manner  that  he  would  search 
through  a  library.  He  has  the  capability  of  browsing 
through  the  collection,  examining  the  documents  filed 

*This  program  is  supported  by  SDCs  independent  research 
funds.  The  principal  investigator  is  Harold  Borko. 
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under  each  subjeet  category,  and  he  is  also  able  to 
search  for  documents  containing  very  specific  informa¬ 
tion.  If  he  is  not  sure  of  the  correct  procedures  to  use, 
he  can  receive  help  and  instructions.  Most  importantly, 
he  is  able  to  state  his  requests  in  natural  English,  for 
the  system  would  surely  fail  if  the  user  first  had  to 
learn  programming  before  he  eould  retrieve  informa¬ 
tion. 

Like  GPDS  and  PLP  II,  BOLD  is  programmed  for 
use  with  SDC’s  Time-Sharing  System.  The  inquiry  sta¬ 
tion  consists  of  a  teletypewriter  and  display  scope  with 
light  pen.  The  programming  system  has  two  major 
modules:  (1)  the  data-base  generator  program,  which 
builds  tables  of  structured  information  from  a  Hollerith 
prestored  magnetic  tape,  and  (2)  the  display  and  re¬ 
trieval  program,  which  retrieves  the  requested  informa¬ 
tion  from  the  structured  data  base  and  displays  it  on 
the  scope.  A  technical  description  of  the  data  base 
generator  and  of  the  display  and  retrieval  subsystems 
has  been  prepared  by  Howard  Burnaugh  (1966)  who 
wrote  the  programs. 

The  data  base  that  is  presently  being  used  was  ob¬ 
tained  from  the  Defense  Documentation  Center  and 
consists  of  abstracts  of  approximately  6000  documents. 
For  experimental  purposes,  a  subset  of  these  documents 
is  used.  The  particular  tape  from  which  the  illustrative 
examples  were  derived  consists  of  the  first  1745  ab¬ 
stracts  and  6883  retrieval  terms.  The  documents  are 
grouped  into  subjeet  categories  organized  according  to 
the  DDC  classification  system.  However,  the  program 
is  flexible,  and  various  classification  systems  and  index¬ 
ing  systems  can  be  used. 

BOLD  is  an  interactive  system,  which  means  that  a 
dialog  is  established  between  the  user  and  the  system 
to  enable  the  user  to  request  and  obtain  relevant  docu¬ 
ments  from  the  collection.  The  requests  and  the  system’s 
responses  are  stated  in  as  elose  an  approximation  to 
natural  language  as  is  possible.  Ideally,  the  user  with 
only  a  knowledge  of  the  English  language  and  a  skill 
in  typing  should  be  able  to  establish  a  rapport  with  the 
machine.  Although  this  ideal  may  never  be  fully 
achieved,  a  great  deal  of  human  engineering  skill  has 
gone  into  the  project  to  approximate  it. 

After  a  user  has  logged  in  and  the  data  base  and 
program  tapes  are  loaded,  the  system  reports  this  fact 
by  typing 

THIS  STATION  IS  NOW  UNDER  THE  CON¬ 
TROL  OF  THE  BOLD  SYSTEM 

OPERATION  INSTRUCTIONS  R  OBTAINED 

BY  THE  REQUEST: 

INSTRUCTIONS/ 

Simultaneously,  a  tutoring  display  (Figure  5)  will  ap¬ 


pear  on  the  scope.  This  display  defines  the  10  light-pen 
actions  that  are  available  to  the  user. 
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Figure  5 — Initial  tutoring  display 

The  user  begins  operation  by  flashing  the  “B”  char¬ 
acter  with  the  light  pen  or  typing  BEGIN/  on  the  tele¬ 
typewriter.  Commands  such  as  BEGIN,  SEARCH, 
BROWSE,  CONTINUE,  etc.,  must  be  followed  by  a 
slash.  The  user  types  a  question  mark  to  ask  for  help, 
and  for  all  other  interactions  no  punctuation  marks 
are  used. 
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Figure  6 — Classification  categories 


When  the  BEGIN  command  is  accepted  (signified 
by  //),  a  new  display  appears  (Figure  6)  that  indicates 
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the  32  divisions  or  main  subject  categories  into  which 
the  data  are  divided.  If  the  user  wishes  a  further  break¬ 
down,  he  may  use  his  light  pen  to  flash  a  division.  By 
doing  so,  lie  is  requesting  more  information  about  that 
category  and  receives  a  display  of  the  subdivisions  and 
the  number  of  entries  in  the  category.  If  he  chooses  to 
browse  through  the  items  in  this  category,  he  docs  so 
by  cither  flashing  the  □  character  with  his  light  pen  or 
typing  BROWSE/  on  the  teletypewriter.  He  then  re¬ 
ceives  a  display  consisting  of  the  first  abstract  in  the 
selected  category  (see  Figure  7).  If  this  abstract  is  not 
complete,  because  of  the  limited  number  of  charac¬ 
ters  that  could  be  displayed  on  the  scope,  he  may 
obtain  the  remainder  by  light-penning  the  “C”  or  “con¬ 
tinue”  character.  In  a  similar  manner,  he  can  view  all 
the  abstracts  in  the  selected  category. 
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Figure  7 — Viewing  lhe  reirieved  abstraci 

Although  browsing  through  an  organized  collection 
of  documents  is  one  way  of  searching  for  information, 
a  more  commonly  used  method  is  to  request  docu¬ 
ments  by  subject  headings  or  index  terms.  Many  in¬ 
formation  centers  use  a  form  of  coordinate  indexing, 
and  retrieve  information  by  combining  a  number  of 
index  terms  to  form  a  specific  request.  Usually  a  trained 
information  specialist  must  help  the  user  formulate  his 
request  for  information  into  a  search  request  made  up 
of  approved  index  terms.  In  an  interactive  system,  the 
user  requests  help  by  interrogating  the  dictionary. 

By  way  of  illustration,  let  us  suppose  the  user  is 
doing  research  in  the  field  of  space  travel.  He  is  pre¬ 
paring  a  report  on  this  subject  and  hc  wishes  to  search 
the  collection  for  relevant  articles.  Hc  sits  at  the  in¬ 
quiry  station,  and  first  interrogates  the  dictionary  to 


determine  which  words  can  be  used  as  index  terms  for 
retrieval  purposes. 

The  following  dialog  takes  place: 

SPACESHIPS? 

THESE  MAY  BE  RELATED  TO  SPACESHIPS 

SPACESHIP  CABINS 

SPACESHIPS 

SPACESHIPS  -  POWER  SUPPLIES 
SPACESHIPS  -  STABILITY 
*END 
SPACE? 

THESE  MAY  BE  RELATED  TO  SPACE 
SPACE  CAPSULES 
SPACE  CHARGES 

SPACE  ENVIRONMENTAL  CONDITIONS 

SPACE  FLIGHT 

SPACE  FLIGHT  -  CONTROL 

SPACE  FLIGHT  -  SURVIVAL 

SPACE  MEDICINE 

♦CONTINUE?  YES 

SPACE  MEDICINE  -  EFFECTIVENESS 
SPACE  NAVIGATION 
SPACE  PERCEPTION 
SPACE  PROBES 

SPACE  RECOVERY  SYSTEMS,  INC.,  EL  SE- 
GUNDO,  CALIF. 

SPACE  SCIENCES  LAB.,  GENERAL  ELECTRIC 
CO.,  PHILADELPHIA,  PA. 

SPACE  SHIPS 
*CONTI  NUE?NO 
LUNAR  FLIGHTS? 

♦NOT  FOUND 
MOON  FLIGHTS 
♦NOT  FOUND 
MARS  FLIGHTS? 

♦NOT  FOUND 
MOON? 

THESE  MAY  BE  RELATED  TO  MOON 
MOON 

MOON  -  ATMOSPHERE 

♦END 

LUNAR? 

THESE  MAY  BE  RELATED  TO  LUNAR 

LUNAR  PROBES 

♦END 

MARS? 

THESE  MAY  BE  RELATED  TO  MARS 
MARS 

MARSH  CHARLES  A. 

MARSHALL  JOHN  M. 

♦END 


Hc  begins  by  asking  whether  SPACESHIPS  is  an 
index  term  by  typing  the  word  followed  by  a  question 
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mark.  The  system  responds  that,  in  addition  to  SPACE¬ 
SHIPS,  there  are  a  number  of  other  similar  terms  that 
are  also  usable  index  words.  The  system  finds  these 
related  terms  by  dividing  the  query  word  in  half  and 
locating  all  index  terms  that  start  with  the  same  com¬ 
bination  of  letters. 

The  user,  now  recognizing  that  the  term  SPACE¬ 
SHIPS  might  be  too  specific,  asks  for  information  about 
the  more  general  term  SPACE.  Again  the  system  re¬ 
sponds  with  a  set  of  related  terms.  Note  that  the  word 
SPACE  by  itself  is  not  an  index  term,  for  it  is  always 
used  in  combination  with  another  word.  In  response  to 
a  dictionary  inquiry,  the  system  types  seven  index  terms 
and  then  asks  the  user  whether  he  wishes  it  to  continue. 
After  two  such  inquiries,  the  user  feels  he  has  enough 
information  on  this  subjeet  and  tries  some  other  terms. 
Some  of  the  words  he  tries  are  not  index  terms,  but  in 
his  interaction  he  finds  enough  that  are. 

As  a  result  of  this  dialog,  and  with  the  information 
he  has  obtained,  he  is  now  in  a  position  to  formulate 
a  search  request.  He  selects  six  terms  and  formulates 
these  into  a  seareh  request  by  indieating  that  he  would 
like  to  have  displayed  the  list  of  document  numbers 
that  contains  any  one  of  these  six  terms;  that  is,  he  com¬ 
bines  these  terms  by  means  of  an  OR  rather  than  an 
AND  logic,  although  both  the  AND  and  NOT  logic 
are  also  available. 

He  makes  his  request  as  follows: 


SPACFSHIPS  OR  LUNAR  PROBES  OR  MOON  OR  MARS 


25 

ENTRIES 

ARE 

REFT) 

BY 

SPACESHIPS 

6 

ENTRIES 

ARE 

REF'D 

BY 

LUNAR  PROBES 

13 

ENTRIES 

ARE 

REFT) 

BY 

MOON 

3 

ENTRIES 

ARE 

REF’D 

BY 

MARS 

♦END 

SPACE  FLIGHT  OR 
15  ENTRIES  ARE 

SPACE 

REF’D 

PROBES 

BY 

SPACE 

FLIGHT 

8 

ENTRIES 

ARF. 

REF’D 

BY 

SPACE 

PROBFS 

♦END 

SEARCH/ 

51  ENTRIES 


Note  that  when  the  user  types  a  request,  as  distinct 
from  interrogating  the  dietionary,  he  does  not  use  a 
question  mark.  The  system  tells  him  how  many  entries 
in  this  data  base  (1745  abstracts)  are  referenced  by 
each  term. 


He  now  orders  the  system  to 
SEARCH/ 

and  the  system  responds  that  there  are 
51  ENTRIES 

Since  there  is  a  total  of  70  documents  that  have  been 
indexed  by  these  six  terms,  it  is  clear  that  some  docu¬ 
ments  were  indexed  by  more  than  one. 

The  system  locates  these  51  documents  and  dis¬ 
plays  the  list  by  identification  number  and  index  term. 

The  display  appears  on  the  scope  (Figure  8).  Note 
that  not  all  the  documents  can  be  displayed  at  one 
time.  Of  the  51  entries  only  37  have  been  searched. 
The  user  may  now  remove  references  to  the  documents 
that  are  of  less  interest  by  light-penning  “RM  and  the 
document  number.  By  light-penning  the  “C,"  or  “con¬ 
tinue'”  character,  he  allows  newf  document  references 
to  be  displayed.  He  may  also  reorder  the  arrangement 
of  the  display  by  light-penning  the  “E”  character  and 
two  document  numbers  that  he  wishes  to  exchange. 
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Figure  8 — Search  matrix  of  retrieved  documents 


Before  requesting  eopies  of  the  51  documents  that 
have  been  indexed  by  one  or  more  of  the  six  retrieval 
terms,  the  user  would  like  to  have  more  information 
about  their  contents.  He  may  obtain  this  information 
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by  simply  typing  BROWSE/  or  by  light-penning  the  ap¬ 
propriate  BROWSE  symbol. 

The  system  responds  to  the  BROWSE  command 
with 

INPUT  ATTRIBUTES  WANTED 
In  this  instance,  let  us  assume  that  the  user  wishes 
to  see  just  the  author  and  title  of  the  retrieved  articles, 
so  he  lists  these  as  the  attributes  wanted.  He  could  also 
have  requested  the  index  terms,  the  contract  number, 
the  date  of  publication,  or  the  complete  abstract. 

The  first  set  of  authors  and  titles  is  displayed  on  the 
seope  (Figure  9)  and  the  rest  can  be  obtained  by  the 
“continue”  action. 


Figure  9 — Document  titles  and  authors  displayed  on  scope 
BOLD 

Should  an  immediate  permanent  record  be  wanted, 
it  can  be  obtained  by  the  command. 

TYPE  DISPLAY/ 

In  this  manner,  the  user  can  browse  through  the  en¬ 
tire  set  of  51  entries  that  have  been  retrieved  in  response 
to  his  request,  or  that  subset  of  documents  that  he  has 
not  removed  from  the  display.  He  may  save  any  infor¬ 
mation  that  appears  for  future  reference.  He  interacts 
with  the  system,  and  when  he  leaves  the  inquiry  station 
he  leaves  with  the  feeling  that  he  has  obtained  most  of 
the  relevant  material  that  the  system  has  in  store.  The 
response  has  been  rapid,  and  the  experience  has  been 
a  satisfying  one. 


CONCLUSION 

Three  of  the  SDC  interactive  programming  systems 
have  been  described.  These  are:  (1)  the  General  Pur¬ 
pose  Display  System  (GPDS),  (2)  the  Pattern  Learn¬ 
ing  Parser  (PLP  II),  and  (3)  the  Bibliographic  On- 
Line  Display  System  (BOLD).  The  versatility  of  these 
systems  is  illustrated  by  the  range  of  problems  that 
they  are  eapable  of  handling.  By  using  on-line  inter¬ 
active  displays,  the  man  and  the  maehine  are  able  to 
engage  in  a  dialog  as  both  work  together  to  solve 
problems.  The  computer  processes  data  rapidly  and  dis¬ 
plays  the  results.  The  human  decision  maker  interprets 
the  displays  and  determines  the  accuracy  and  relevance 
of  the  results.  The  information  provided  in  the  displays 
enables  him  to  steer  and  eontrol  the  step-by-step  prog¬ 
ress  of  the  program.  As  a  result  of  his  involvement, 
problems  are  solved  more  efficiently  and  in  a  more 
satisfying  manner. 
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evidence  by  man-machine  systems* 
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INTRODUCTION 

Two  years  ago  at  this  same  gathering  George  Briggs  and 
I  described  some  of  the  details  and  early  results  of  the 
man-maehine  system  simulation  program  at  The  Ohio 
State  University  Human  Performance  Center  (formerly 
ealled  the  Laboratory  of  Aviation  Psychology).  This 
simulation  program  was  then  and  still  is  concerned  with 
human  performance  in  information-processing  systems 
which  perform  diagnostic  functions.  The  initial  impetus 
for  examining  man’s  role  in  diagnostic  systems  came 
from  papers  by  Ward  Edwards  (1962,  1963).  It  be¬ 
came  apparent  at  the  time  the  first  of  these  papers 
appeared  that  the  man-maehine  system  simulation  facil¬ 
ities  being  developed  at  OSU  would  be  rather  ideally 
suited  for  research  on  some  of  the  issues  raised  in 
Edwards’  paper.  Since  1962  a  fairly  large  number  of 
experiments  have  been  performed  using  these  system 
simulation  facilities.  Detailed  descriptions  of  the  simula¬ 
tion  facilities  arc  available  elsewhere  (Brings  &  Schum, 
1965;  Schum,  Goldstein  &  Southard,  1965a).  Some  of 
the  research  has  been  described  in  reports  issued  through 
our  sponsoring  agency  (Southard,  Schum,  &  Briggs, 
1964a,  b;  Schum,  1966b;  Schum,  Goldstein,  &  South¬ 
ard,  1965a,  b).  Other  research  has  been  reported  in 
various  professional  journals  (Schum,  1966a,  b;  Schum, 
Goldstein,  &  Southard,  1966;  Schum,  Goldstein,  How¬ 
ell,  &  Southard,  1966).  Both  the  development  of  the 
system  simulation  facility  and  the  subsequent  research 
program  have  been  sponsored  by  the  Behavioral  Scien¬ 
ces  Laboratory,  Aerospaee  Medieal  Research  Labora¬ 
tories,  United  States  Air  Foree  Systems  Command. 

There  have  been  two  major  objectives  in  our  re- 

*Thc  research  reported  in  this  paper  was  sponsored  by  the 
Aerospace  Medical  Research  Laboratories,  Aerospace  Medical 
Division,  Air  Force  Systems  Command,  Wright-Patterson  Air 
Force  Base,  Ohio,  under  Contract  No.  AF  33(615  >-2248  with 
the  Ohio  State  University  Research  Foundation.  Further  re¬ 
production  is  authorized  to  satisfy  the  needs  of  the  United 
States  Government. 


seareh  program.  The  first  objective  has  been  to  provide 
information  about  human  capabilities  and  limitations  in 
the  tasks  of  evaluating  and  aggregating  probabilistic 
evidence  in  complex  circumstances.  The  second  ob¬ 
jective  has  been  to  uncover  and  investigate  some  of  the 
problems  which  might  be  encountered  if  computer- 
implemented  Bayesian  evidence-aggregation  procedures 
were  used  in  actual  situations.  Several  changes  have 
been  made  in  our  simulation  methodology  in  the  two 
years  sinee  our  last  report.  Much  of  the  detail  in  sim¬ 
ulating  a  real-time  stimulus  environment  was  found  to 
be  unnecessary  for  current  purposes.  In  addition,  sev¬ 
eral  lower-level  information-processing  functions  per¬ 
formed  by  men  in  our  earlier  experiments  were  not 
incorporated  in  the  current  series  of  experiments.  In 
the  present  paper  I  will  describe  some  of  the  features  of 
our  present  research  on  man's  role  in  diagnostic  systems 
and  will  discuss  the  manner  in  which  system  perform¬ 
ance  criterion  problems  have  affected  our  efforts. 

Man's  role  in  diagnostic  systems 

The  output  of  a  diagnostic  system  is  in  the  form  of 
an  opinion  about  the  likelihood  of  each  of  the  various 
hypothesized  eauses  or  states  of  the  world  which  are 
assumed  to  be  underlying  the  emergence  of  data  from 
some  environment  of  interest  to  the  system.  Revisions 
of  these  opinions  are  required  as  new  data  emerge. 
Conceptually,  at  least  two  basic  information-processing 
steps  are  required.  The  first  step  involves  evaluating 
the  diagnostic  impaet  of  items  of  evidence  as  they  oeeur. 
This  is  roughly  equivalent  to  determining  the  extent  to 
which  each  datum  allows  discriminations  to  be  made 
among  the  hypotheses.  In  some  instances,  historical 
records  in  the  form  of  relative  frequencies  linking  data 
and  hypotheses  are  available  as  aids  in  this  process. 
More  often,  perhaps,  the  assessment  of  evidence  impact 
is  possible  only  by  means  of  the  judgments  of  some  ex¬ 
pert  whose  intuition  has  been  educated  through  experi¬ 
ence  with  similar  or  related  phenomena.  Evaluation  of 
unique  one-of-a-kind  military  intelligence  data  is  an 
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example  of  this  sort  of  impact  assessment.  The  second 
basic  information-processing  task  involves  aggregation 
of  the  impact  of  each  new  evidence  item  with  the 
combined  impact  of  all  items  previously  evaluated.  In 
other  words,  current  opinions  about  the  likelihood  of 
hypotheses  result  from  an  aggregation  of  new  data  with 
old  data.  The  state  of  the  system’s  current  opinions 
can  be  indicated  by  posterior  probabilities,  posterior 
odds,  or  any  other  similar  expression  of  confidence  in 
the  truth  of  each  of  the  hypotheses. 

By  now,  nearly  everyone  with  an  interest  in  man- 
machine  information-processing  systems  is  probably 
aware  of  Edwards’  proposals  for  the  allocation  of  the 
above  tasks  among  the  man  and  machine  components 
of  diagnostic  systems.  If  men  can  make  evidence  im¬ 
pact  assessments  in  the  form  of  P(D  H),  likelihood 
ratio,  or  some  other  analogous  quantity,  then  computers 
programmed  according  to  Bayes’  theorem  could  per¬ 
form  the  requisite  aggregations.  Bayes’  theorem  pre¬ 
scribes  how  the  aggregation  should  be  performed.  Sev¬ 
eral  recent  experiments  have  demonstrated  that,  in 
diagnostic  or  inferential  tasks,  men  extract  a  signifi¬ 
cantly  decreasing  proportion  of  total  amount  of  diag- 
nostieity  in  data  as  the  amount  of  data  to  be  evaluated 
are  increased  (Peterson,  Schneider,  &  Miller,  1965; 
Sehum,  1966a;  Sehum  et  al.,  1966).  Bayesian  evidence- 
aggregation  procedures  ought  to  curtail  the  loss  in 
diagnostieity  which  results  from  limitations  in  human 
ability  to  aggregate  large  amounts  of  probabilistic  data. 
These  procedures  should,  therefore,  permit  utilization 
of  large  amounts  of  data  and  should  allow  for  the  in¬ 
corporation  of  more  expert  opinion  in  the  actual  diag¬ 
nostic  process.  Presumably,  several  or  many  experts, 
each  knowledgeable  about  certain  classes  of  data,  would 
provide  the  evidence  impact  assessments.  In  most  cur¬ 
rent  diagnostic  situations  an  individual  assesses  the 
impact  of  evidence  and  performs  the  required  aggrega¬ 
tion  mentally.  The  evidence  he  receives  may  have  been 
altered  through  summarizing  and  editing  operations  by 
members  of  his  staff.  If  there  exist  more  data  than  the 
individual  can  incorporate  in  his  judgments,  he  must 
ignore  or  discard  some  of  them.  This  is  a  wasteful 
procedure  (some  or  all  of  the  discarded  data  may  have 
been  obtained  at  great  expense),  and  it  may  be  a  dis¬ 
astrous  procedure  if  the  individual  ignores  or  discards 
data  which  happen  to  conflict  with  prior  expectations 
he  might  have  of  the  truth  of  certain  hypotheses.  There 
are  thus  at  least  two  methods  of  producing  output  in 
systems  which  perform  diagnostic  functions.  Posterior 
probabilities  or  odds  can  be  estimated  by  men  directly 
from  the  data  or  they  can  be  calculated  by  Bayesian 
aggregation  of  men’s  estimates  of  P(D|H)  or  some 
related  quantity.  There  may,  of  course,  be  other  meth¬ 
ods  of  arriving  at  expressions  of  confidence  in  the  truth 


of  hypotheses.  Moreover,  there  may  be  diagnostic  situa¬ 
tions  in  which  basic  information-processing  function^, 
other  than  those  described  above,  are  required. 

Concerning  performance  criteria  in  diagnostic  system 

simulation 

Diagnoses  form  the  basis  for  selection  of  a  course 
of  action.  Rational  selection  of  a  course  of  action  de¬ 
mands  thorough  consideration  of  the  cost  and  payoff 
effects  that  the  alternative  actions  may  produce.  It  is 
possible,  as  Edwards  (1966)  has  recently  explained, 
to  evaluate  the  diagnostic  process  independently  from 
the  action-selection  process.  There  are,  apparently, 
some  unique  problems  in  evaluating  inferential  or 
diagnostic  behavior  regardless  of  the  context  in  which 
such  behavior  occurs  and  regardless  of  the  method  by 
which  the  diagnoses  are  produced.  The  most  obvious 
criterion  for  evaluating  inference  accuracy  involves  the 
question  of  whether  or  not  the  hypothesis  identified  as 
most  probable  by  some  inference  method  (just  prior 
to  the  selection  of  some  action)  was  in  fact  the  “true” 
hypothesis,  i.e.,  the  hypothesis  which  actually  gen¬ 
erated  the  observed  evidence.  This  criterion,  although 
conceptually  simple,  presents  several  difficulties.  First, 
there  may  be  no  way  to  verify  which  hypothesis  actually 
generated  the  evidence.  Sueh  verification  may  depend 
upon  additional  data  which  the  involved  system  may 
not  be  able  to  collect.  One  can  always  argue  that  subse¬ 
quent  evidence,  even  if  obtainable,  can  only  strengthen 
or  diminish  one’s  confidence  in  previously  selected 
hypotheses  but  cannot  absolutely  verify  them.  External 
sources  from  which  one  might  seek  verification  of  hypo¬ 
theses  may  be  difficult  to  find. 

Another  problem  associated  with  the  above-men¬ 
tioned  criterion  arises  because  of  the  equivocal  nature 
of  evidence.  Suppose  that  the  true  hypothesis  (hereafter 
symolized  as  Ht)  is  subsequently  known  or  verified  in 
some  manner  to  everyone’s  satisfaction.  A  diagnostician 
may  have  assigned  entirely  appropriate  impact  to  the 
evidence  he  has  observed  and  he  may  have  aggregated 
this  evidence  in  a  near-optimal  manner.  Yet  he  could 
be  led  by  the  evidence  toward  acceptance  of  the  wrong 
hypothesis.  In  a  word,  an  improbable  series  of  events 
may  have  happened  and,  although  behaving  in  a  highly 
rational  manner,  the  diagnostician  was  led  by  the  evi¬ 
dence  toward  the  wrong  hypothesis.  (1  am  aware  of 
the  fact  that  the  state  of  this  diagnostician's  prior  ex¬ 
pectations  about  the  truth  of  each  of  the  hypotheses  also 
influences  which  hypothesis  he  might  ultimately  have 
accepted.)  The  point  is  that  a  performance  criterion 
based  solely  upon  Ht  may,  on  occasion,  be  insensitive 
to  ideal  behavior.  Because  all  evidence  is,  to  some  de¬ 
gree,  consistent  with  the  truth  of  more  than  one  hypoth¬ 
esis,  it  is  necessary  to  emphasize  that  even  Bayesian 
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evidence-aggregation  procedures  guarantee  only  logical 
consistency  in  combining  evidence  and  not  perfect  ac¬ 
curacy  in  the  selection  of  Ht. 

There  is,  finally,  uncertainty  associated  with  the 
specification  of  an  exhaustive  set  of  mutually  exclusive 
hypotheses.  One  can  allow  for  this  uncertainty  but  not 
reduce  it  by  defining  a  single  diffuse  hypothesis  which 
represents  the  union  of  those  unknown  or  obscure 
causes  or  states  of  the  world  one  might  not  have  con¬ 
sidered.  This  single  diffuse  conglomerate  of  unknown 
causes  is  euphemistically  termed  the  “catch-all  hypoth¬ 
esis.”  The  problem  relates  to  the  level  of  description 
one  desires  in  formulating  hypotheses.  One  can  always 
dichotomize  states  of  the  world  into  some  gross  state 
(such  as  “war”)  and  its  complement  (“not  war”  or, 
perhaps,  “peace”).  Selection  of  an  effective  course  of 
action,  however,  requires  a  more  subtle  description  of 
the  state  of  the  world.  If  war  is  imminent,  we  should 
at  least  want  to  specify  who  our  adversaries  are.  In  a 
more  subtle  formulation  of  hypotheses  one  may,  through 
ignorance,  leave  out  valid  potential  explanations  for 
the  evidence  he  will  collect.  Some  protection  is  af¬ 
forded  by  a  catch-all  hypothesis.  In  most  diagnostic 
situations  one  should  be  prepared  to  reformulate  one's 
hypothesis  set  as  evidence  appears.  This  presents  further 
difficulties  for  performance  criteria  based  upon  H,.  The 
true  explanation  for  some  pattern  of  evidence  may 
never  have  been  explicitly  included  in  one's  set  of 
explanatory  hypotheses. 

Having  all  but  damned  criterion  measures  based 
upon  H,  as  probably  unobtainable  and  frequently  in¬ 
sensitive  in  actual  inference  situations,  I  shall  proceed 
to  explain  why  and  how  we  have  used  such  measures 
in  our  simulation  research  over  the  past  four  years. 
We  sought  to  evaluate  human  commerce  with  the  con¬ 
ditional  probabilities  implied  by  the  Bayesian  paradigm 
in  an  environment  which  would  be  complex  but  con¬ 
trollable,  one  in  which  a  wide  range  of  variables  of 
interest  to  diagnostic  systems  could  be  manipulated. 
It  was  the  issue  of  controllability  which  led  us  to  the 
type  of  data-generation  model  used  (with  some  varia¬ 
tion)  in  all  of  our  studies  The  data-generation  model 
is  the  set  of  underlying  contingency  rules  which  gov¬ 
erned  the  appearance  of  and  relationships  between  the 
events  upon  which  subjects  based  their  probability 
estimates.  All  of  the  models  used  in  our  various  experi¬ 
ments  were  matrices  of  experimenter-defined  objective 
probabilities  of  the  form  P(D  H).  These  P(D  H) 
values  define  the  probability  of  occurrence  of  each  pos¬ 
sible  event  under  each  of  a  specified  exhaustive  set  of 
mutually  exclusive  hypotheses.  The  hypotheses  and 
data  described  causes  and  effects  in  a  fictitious  hostile 
environment. 

Data  existed  in  major  classes,  each  class  having  an 


exhaustive  set  of  mutually  exclusive  subclasses.  Dis¬ 
tributions  of  P(D  H)  were  assigned  by  the  experi¬ 
menters  across  the  subclasses  of  each  data  class  under 
each  hypothesis.  Particular  values  of  P(D  H)  assigned 
in  this  manner  depended  upon  the  objectives  of  the 
experiment  in  question.  Samples  of  evidence  of  “scenar¬ 
ios”  were  generated  by  first  choosing  some  hypothesis 
and  then  generating  a  single  subclass  from  each  data 
class  according  to  the  multinominal  P(D  H)  distribu¬ 
tions  appropriate  to  each  class  and  to  the  hypothesis 
in  question  (Ht).  The  essential  fact  is  that  we,  as 
experimenters,  were  in  the  position  of  knowing  what 
Ht  was  for  each  sample  of  evidence  thus  generated. 

True  P(D  H)  values  were  defined  according  to  the 
(sometimes  bizarre)  imagination  of  the  experimenters. 
Subjects,  therefore,  could  not  be  expected  to  make  rea¬ 
sonable  estimates  of  P(D  H)  except  through  repeated 
exposure  to  events  in  the  environment.  The  basic  fre¬ 
quent  is  tic  character  of  our  quasi-realistic  data  base  is 
thus  apparent.  Subjects  could  learn  evidence  impact  in 
the  form  of  P(D]H)  only  by  accumulating  relative 
frequencies  linking  data  and  hypotheses.  The  manner 
in  which  subjects  could  determine  evidence  impact 
forms  the  basic  methodological  difference  between  our 
systems  experiments  and  those  of  Ward  Edwards  and 
his  associate  (1966)  and  those  of  Kaplan  and  New¬ 
man  (1966).  Edwards  and  his  associates  have  developed 
a  facility  for  presenting  unique,  one-of-a-kind  or  non- 
frequcntistic  intelligence  data  to  subjects.  Evidence  im¬ 
pact  was  specified  by  Edwards'  subjects  on  the  basis 
of  plausible  linkages  of  data  and  hypotheses.  Subjects 
could  form  these  linkages  because  they  had  extensive 
training  in  the  description  and  past  history  of  a  quasi- 
rcalistie  environment.  Kaplan  and  Newman  used  a 
data-generation  model  based  upon  a  circular  normal 
probability  distribution.  There  is  a  good  reason  for 
questioning  the  suitability  of  a  frequentistic  environ¬ 
ment  for  studying  human  ability  to  make  assessments 
of  evidence  impact  in  the  form  of  P(D  H)  estimates. 
One  can  argue  that,  if  relative  frequencies  linking  data 
and  hypotheses  were  available  in  some  situation,  further 
estimation  of  P(D  H)  by  men  would  be  unnecessary. 
I  hope,  however,  to  be  able  to  demonstrate,  in  the  next 
section,  that  meaningful  experiments  can  be  performed 
regarding  data  impact  assessments  when  a  frequentistic 
data  base  is  used. 

In  our  simulation  experiments  we  have  been  able 
to  specify  the  exact  hypothesis  under  which  each  evi¬ 
dence  sample  was  generated.  In  addition,  we  could 
easily  specify  a  prior  probability  distribution  by  means 
of  which  we  could  regulate  the  number  of  times  a 
given  hypothesis  would  be  true  during  the  course  of  an 
experiment.  In  other  words,  specified  prior  probability 
distributions  regulate  the  number  of  scenarios  gen- 
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crated  under  each  hypothesis.  All  of  the  several  de¬ 
pendent  variables  used  in  our  experiments  to  indicate 
diagnostic  accuracy  have  been  based  upon  our  knowl¬ 
edge  of  Ht  for  each  scenario.  In  addition,  all  of  these 
dependent  variables  have  involved  posterior  probabili¬ 
ties  or  operations  performed  upon  posterior  probabil¬ 
ities.  In  what  follows,  P(Ht  D)  will  indicate  the  value 
of  the  posterior  probability  (estimated  or  calculated) 
which  was  placed  under  H(.  In  addition,  I  will  be  re¬ 
ferring  to  terminal  values  of  P(Ht|D)  or  P(H,JD) 
estimated  or  calculated  on  the  basis  of  all  data  in  some 
scenario  (D  thus  indicates  the  evidence  in  an  entire 
scenario).  There  arc  several  methods  of  producing 
P(H,  D)  in  our  experiments.  The  objectives  of  the 
particular  experiment  being  performed  determine  which 
methods  can  be  compared.  Following  is  a  list  of  the 
basic  methods  for  determining  P(H,|D)  in  our  experi¬ 
ments.  The  P(H,|D)  values  resulting  from  all  of  these 
methods  rest  upon  assumption  of  the  prior  probability 
distributions  specified  by  the  experimenter. 

1.  P(H,|D)  can  result  from  estimates  made  directly 
from  data  by  subjects  when  (a)  the  evidence  impact 
has  been  specified  for  the  subjects  by  the  experimenter, 
or  (b)  the  evidence  impact  is  learned  by  subjects 
through  repetitive  instances  in  which  data  are  known  to 
have  occurred  in  the  presence  of  the  various  h>pothcses 
(this  learning  requires  some  subsequent  knowledge  of 
Ht  by  subjects).  If  evidence  impact  is  specified  by  the 
experimenter,  the  only  task  for  subjects  is  evidence 
aggregation. 

2.  P(H,|D)  can  be  calculated  from  Bayes’  theorem 
using  subjects’  estimates  of  P(D  H)  or  some  related 
quantity  (the  precise  forms  that  these  "related  quan¬ 
tities"  can  take  are  specified  in  the  next  section  of  this 
paper).  Evidence  impact  in  the  form  of  P(D  H)  is 
learned  through  accumulation  of  relative  frequencies. 
More  complicated  impact-estimation  tasks,  however, 
require  some  aggregation  of  probabilities.  A  discussion 
of  these  more  complicated  tasks  is  also  included  in  the 
next  section.  This  method  of  posterior  probability  deter¬ 
mination  is  our  analog  of  Edwards’  PIP  (probabilistic 
information  processing)  system  (Edwards,  1962). 

3.  P(Ht]D)  can  be  calculated  from  the  data  in  each 
scenario  using  Bayes’  theorem  and  the  “true’’  experi¬ 
menter-specified  P(DjH)  values.  This  calculation  of 
P(H/D)  results  from  the  best  possible  aggregate  of 
items  of  evidence  whose  impacts  are  perfectly  known. 
It  is  thus  the  theoretically  ideal  P(Ht|D).  This  quantity 
is  highly  useful  in  scaling  levels  of  such  variables  as 
degree  of  conditional  nonindependence  among  evidence 
items,  scenario  diagnosticity,  and  degree  of  evidence 
conflict.  This  calculation  also  serves  as  a  model  for 
ideal  inference  behavior  when  subjects  themselves  have 
access  to  true  P(D|H)  as  in  (la)  above.  This  calcula¬ 


tion  can  be  performed  under  the  assumption  that  the 
data  in  a  scenario  arc  mutually  independent  under  every 
hypothesis.  It  can  also  be  performed  upon  conditionally 
nonindependent  data.  Such  calculation  requires  that  the 
experimenter  “tell"  Bayes’  theorem  where  the  nomin- 
dependcncc  exists.  This  is  done  by  incorporating  joint 
conditional  probabilities  of  the  form  P(DX,D,'H)  in 
the  Bayesian  solution. 

4.  P(H,  D)  can  be  calculated  from  the  current  rela¬ 
tive  frequency  estimates  of  P(D]H]D)  [or  P(DX,D  H)] 
at  the  time  the  scenario  was  presented.  A  requirement 
is  that  the  computer  be  programmed  so  that  the  relative 
frequency  estimates  of  these  quantities  can  be  updated 
in  the  computer  as  each  new  event  occurs  during  some 
experiment.  This  calculation  represents  a  model  for 
optimal  inference  behavior  when  the  impact  of  evi¬ 
dence  must  be  learned  by  the  subjects,  as  in  (lb,  or 
(2)  above.  It  incorporates  the  same  information  avail¬ 
able  to  the  subjects  at  every  point  in  time  during  an 
experiment. 

The  Bayesian  calculations  described  in  (3)  and  (4) 
above  are,  of  course,  performance  criteria  in  them¬ 
selves.  In  one  sense,  Ht  merely  provides  a  reference 
point  for  comparisons  of  subjects’  responses  in  methods 
(la,b)  and  (2)  with  calculations  in  (3)  and  (4).  One 
problem  encountered  in  these  comparisons  arises  when 
some  scenario  contains  a  large  proportion  of  data  most 
highly  consistent  with  the  truth  of  some  hypothesis 
other  than  the  one  under  which  the  scenario  was  gen¬ 
erated.  When  this  happens,  calculated  P(Ht  D)  may 
not  be  the  largest  value  in  the  terminal  posterior  proba¬ 
bility  distribution.  Such  a  problem  is  generally  en¬ 
countered  in  our  work  when  scenario  size  is  small. 

A  characteristic  of  our  method  of  generating  scenar¬ 
ios  is  that,  as  the  amount  of  evidence  in  a  scenario  is 
increased,  the  more  likely  it  is  that  Bayesian  terminal 
P(H,  D)  will  be  the  largest  value  in  the  distribution 
of  terminal  posterior  probabilities.  In  other  words, 
scenario  size  and  scenario  diagnosticity  arc  confounded; 
evidence  typically  mounts  in  favor  of  the  hypothesis 
which  is  actually  generating  the  scenario.  Suppose  that 
for  some  scenario  a  subject's  estimate  of  P(Ht  D)  is 
.75  and  some  Bayesian  calculation  of  P(Ht|D)  is  .15. 
Suppose  further  that  there  arc  five  hypotheses.  This 
implies  that  there  is  a  value  of  calculated  P(H  D)  larger 
than  .15  under  some  hypothesis  other  than  Ht.  Suppose 
that  thlis  other  value  is  .70.  In  this  case  there  exists  a 
disparity  between  the  criterion  of  experimenter-known 
“truth”  and  a  Bayesian  criterion  representing  the  out¬ 
come  of  formally  optimal  evidence  aggregation.  In 
other  words,  the  scenario  evidence,  in  the  aggregate,  is 
more  highly  consistent  with  the  truth  of  some  hypoth¬ 
esis  other  than  H,.  When  such  a  situation  occurs 
(and  it  sometimes  docs  during  the  course  of  an  experi- 
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nient)^  the  experimenter  is  placed  in  a  logical  cleft 
stick  in  evaluating  the  subjects'  inference  behavior. 
According  to  the  “truth"  criterion  by  itself,  the  sub¬ 
jects’  performance  is  commendable.  By  the  Bayesian 
criterion,  the  subject  has  “backed  into"  the  truth  by 
means  of  inappropriate  interpretation  or  aggregation 
of  the  evidence. 

In  our  experiments  we  have  used  an  assortment  of 
dependent  variables.  One  measure  of  inference  per¬ 
formance  used  in  our  early  experiments  was  simply 
mean  P(llt[D).  Ward  Edwards  then  convinced  us  of 
the  superiority  of  log  odds  and  log  likelihood  ratio.  In 
most  of  our  systems  experiments,  cither  four  or  six 
mutually  exclusive  and  exhaustive  hypotheses  have 
generated  evidence.  In  this  multihypothcsis  situation, 
log  likelihood  ratio  (LLR)  has  been  calculated  by 
means  of  the  following  equation 

log  fit,  —  log  ib  =  LLR  ( I  ) 

where: 

lh  the  posterior  odds  favoring  Ht,  i.e., 

im  D) 

I  P(H,D) 

[1— P(H  D)is  the  sum  of  the  terminal  posterior  prob¬ 
abilities  assigned  under  all  hypotheses  other  than  Ht.] 

fit  =  the  prior  odds  of  Ht,  be.,  _  _ 

1  — P(Ht) 

[1  P(H,)is  the  sum  of  the  prior  probabilities  assigned 
under  all  hypotheses  other  than  Ht  | 

If  LLR  is  calculated  from  subjects’  estimates  of 
P(H  D),  the  resulting  quantities  are  called  subjects’ 
LLR  (SLLR).  When  calculated  from  the  P(Ht  D) 
resulting  from  Bayesian  aggregation  of  subjects’ 
P(D  H)  estimates,  the  LLR  values  are  called  P1PL1  R. 
When  calculated  from  Bayes’  theorem  as  in  methods 
(3)  or  (4)  above,  the  resulting  values  are  called 
Bayes'  LLR  (BLLR).  Two  performance  indices  relat¬ 
ing  SLLR  (or  P1PLLR)  to  BLLR  have  been  used  in 
several  recent  experiments.  One  index  is  called  “ac¬ 
curacy  ratio"  (Peterson,  et  al.,  1965)  where: 

SI  I  R 

Accuracy  Ratio  (AR)  _  (2) 

BLLR 

SLLR  and  BLLR  can  be  either  positive  or  negative 
quantities  which  means  that  AR  can  be  positive  or 
negative.  A  positive  value  of  BLLR  or  SLlR  occurs 
when  ib,  >  ib  ,  i.e.,  when  some  increment  of  cer¬ 
tainty  in  the  truth  of  Ht  has  been  added  to  that  existing 
before  the  occurrence  of  the  evidence.  Negative  BLLR 
or  SLLR  occurs  when  fb]  <  Ht  i.e.,  when  the  evi- 
cvidcnce  occasions  an  opinion  change  in  the  direction 
of  some  hypothesis  other  than  H  with  a  concomitant 
reduction  of  certainty  in  the  truth  of  Ht  from  that  cer¬ 
tainty  existing  before  the  evidence  was  presented.  As 
the  method  of  calculation  of  fb  and  fbo  shown  im¬ 
mediately  below  Equation  1  indicates,  we  arc  actually 


considering  only  two  hypotheses,  Ht  and  its  com¬ 
plement  H,  This  dichotomization  of  the  hypothesis 
set,  the  fact  that  a  few'  scenarios  contain  a  high  propor¬ 
tion  of  evidence  not  favoring  Ht,  and  the  fact  that  sub¬ 
jects  inappropriately  interpret  and  aggregate  evidence 
diagnosticity  in  our  complex  situation  all  combine  in 
such  a  fashion  that  AR,  when  calculated,  is  often  dif¬ 
ficult  to  interpret.  In  addition,  when  ARs  are  com¬ 
bined  across  scenarios  and  subjects,  they  are  often 
misleading. 

Table  1  illustrates  the  various  interpretations  AR 
can  have  in  different  regions  of  its  domain.  AR  is 
actually  indicative  of  the  direction  and  the  amount  of 
subjects’  opinion  revisions  relative  to  those  of  Bayes’ 
theorem.  Positive  ARs  result  when  subject  and  Bayes' 
theorem  move  opinions  in  the  same  direction  (either 
toward  Ht  or  toward  its  complement  Ht).  Negative  AR 
occurs  when  a  subject  moves  his  opinion  in  one  direc¬ 
tion  and  Bayes'  theorem  moves  its  “opinions"  in  the 
other.  In  Table  I,  A,  refers  to  the  amount  of  opinion 
change  by  a  subject  toward  H,  or  H,,  and  A„  refers  to 
the  amount  Bayes’  theorem  opinions  change  toward 
H.  or  Ht. 

Table  I 

Interpretations  of  accuracy  ratio  (AR)  throughout 
its  domain 

 Interpretation 

Agreement  as 
to  Direction 

U<  Run  >e  (Toward  Ht  Relationship  betw  een 

or  1  alue  or  Bt)  As  and  An 


AR  >  1.0 

Yes 

AS>A» 

AR  =  1.0 

Yes 

A<=AU  (Optimality) 

0<AR<  1 .0 

Yes 

A><Ah  (The  conservatism 
effect) 

AR  —  0 

Am>0  Snbjccl  made  no  revi- 

.  sion  of  prior  probabili- 

^  lies  as  a  rcsull  of  the 
evidence. 

—  1  <  AR<() 

No 

As<An  e.g..  subject  moved  op¬ 
inions  more  slowly  lo- 
ward  H,  than  linycs* 
ihcorem  moved  its  op¬ 
inions  ioward  H(. 

> 

73 

II 

l 

b 

No 

As=  Ah  Same  extern  of  opinion 
revision,  differeni  di¬ 
rection. 

AR<  —  1 .0 

No 

As>A»  c.g.,  subject  moved 
opinions  faster  toward 
FT,  than  Hayes’  theorem 
moved  its  opinions  Io¬ 
wa  rd  H,. 

Another  dependent  variable  has  been  termed  the 
“difference  measure"  (Schum,  Goldstein,  Howell,  & 
Southard,  1966),  where: 


Difference  Measure  (D)  =  SLLR  —  BLLR(3) 

In  all  of  our  experiments,  subjects  estimating  P(H  D) 
are  required  to  record,  before  evaluating  data  from 
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any  scenario,  the  prior  probability  distribution  they 
are  actually  assuming.  In  rare  cases  do  these  overt 
expressions  of  prior  probability  differ  from  the  prior 
distributions  specified  for  them  by  the  experimenter 
and  used  by  him  in  calculations  of  P(H|D).  If  these 
reported  priors  can  be  believed,  Equation  3  can  be  re¬ 
written  as: 


D  =  logio  Htl  (subject)  —  log10nto  (Bayes) 

(4) 

or  as: 


D  = 


*(H,|D)  1 

—  *(H,|D)J 


logio 


r  p(h,|d)  i 

Ll  —  P(H,|D)J 
(5) 


where: 

i//(Ht|D)  =  a  subject’s  reported  terminal  posterior 
probability  under  Ht. 

P(Ht|D)  =  the  calculated  value  of  the  terminal  pos¬ 
terior  probability  of  Ht. 

Under  the  assumption  that  a  subject’s  prior  distribu¬ 
tion  is  the  same  as  the  prior  distribution  used  in  Bay¬ 
esian  calculations,  D  is  simply  a  measure  of  the  degree 
of  similarity  between  subjects’  and  Bayes’  theorem 
posterior  odds.  A  D  value  of  zero  occurs  when  lltl 
(subject)  =  Otl  (Bayes),  or  equivalently,  when 
HJD)  =  P(Ht|D).  Negative  D  values  result  when 
Otl  (subject)  <  ntj  (Bayes).  This  implies  that 
<MH,|D)  <  P(Ht|D).  Positive  D  values  occur  when 
ntl  (subject)  >  fltl  (Bayes),  i.e.,  when  ^(Ht|D) 
>  P(Ht|D). 

In  Table  II  are  two  examples  of  AR  and  D  values 
along  with  the  prior  and  posterior  probability  distribu¬ 
tions  from  which  they  were  calculated.  The  subject’s 
estimates  of  posterior  probabilities  and  the  Bayesian 
distributions  are  actual  examples  taken  from  one  of 
our  inference  experiments.  The  most  typical  or  most 
frequently  occurring  values  of  AR  in  an  experiment 


Table  II 

Accuracy  ratio  (AR)  and  difference  measure  (D) 
examples 


EXAMPLE  A 

H, 

Hypotheses:  A 

B 

C 

D 

E 

F 

Priors:  .100 

Subject’s 

.200 

.100 

.500 

.050 

.050 

Terminal  i//(HlD):  .520 
Bavcsian 

.400 

.010 

.050 

.010 

.010 

Terminal  (P(H|D):  .103 

.385 

.014 

.490 

.004 

.005 

<KHt  D)  =  .050;  1  -  i//(Ht|D)  =  .950 
P(Ht  D)  =  .490;  1  —  P(Ht|D)  =  .510 
SLLR  =  —1.2787;  BLLR  =  —.0174 
AR  —  73.49;  D  =  —1.2613 

EXAMPLE  B 

Ht 

A  B  C  D  E  F 


Priors: 

Subject’s 

Terminal  i//(H[D) : 
Bayesian 

Terminal  P(H|D): 
i//(Ht|D)  =  .400; 
P(Ht|D)  =  .164; 
SLLR  =  .521; 

AR  =  =  54.91; 


.167 

.167 

.167 

.167 

.166 

.166 

,050 

.400 

.100 

.200 

.200 

.050 

024 

.164 

.477 

.118 

.168 

.050 

1  — 

>(H, 

D) 

=  .600 

1  = 

P(H, 

D) 

=  .836 

BLLR 

—  — 

-  .0095 

D  = 

.5121 

are  those  in  the  range  0  <  AR  <  1.0.  In  nearly 
every  one  of  our  experiments,  however,  several  aber¬ 
rant  values  of  AR,  such  as  those  in  Table  II,  have 
occurred.  In  Example  A  large  positive  AR  (73.49) 
reflects  the  fact  that  both  subjects  and  Bayes’  theorem 


opinions  moved  away  from  Ht  (Hypothesis  D)  on 
the  basis  of  scenario  evidence.  The  Bayesian  move¬ 


ment,  however,  was  small  relative  to  the  subject’s 
movement.  The  negative  value  of  D  in  Example  A 
indicates  that  the  subject’s  posterior  odds  under  Ht 
(5/95)  were  small  compared  with  Bayesian  posterior 
odds  (49/51 ).  Antilog  D  is  the  ratio  of  fltl  (subject)  to 
Hti  (Bayes). 

In  this  example 

ntj  (subject)  1 

nt]  (Bayes) 

In  example  B  Bayes’  theorem  is  led  away  from  Ht  (Hy¬ 
pothesis  B)  by  a  small  amount,  yet  the  subject  has 
moved  his  opinions  in  the  direction  of  Ht  by  a  sub¬ 
stantial  amount.  The  disparity  in  direction  of  move¬ 
ment  produces  the  negative  AR  value.  The  large  size 
of  the  AR  value  ( — 54.91 )  is  due  to  the  large  difference 
in  the  amount  by  which  the  subject  and  Bayes’  theorem 
moved  their  opinions.  The  positive  D  value  (.5121) 
results  from  llx  (subject)  being  greater  than  fltl 

(Bayes).  Antilog  D  =  (subject)  _  3  25 

fttj  ( Bayes) 

A  fourth  dependent  variable,  called  a  “dichotomous 
measure,”  ignores  the  absolute  size  of  i//(Ht|D)  or 
P(Ht  D).  Using  this  dichotomous  measure,  one  asks 
only  whether  or  not  i^(Ht|D)  [or  P(Ht!D)]  was  the 
largest  value  in  some  posterior  probability  distribution. 
There  are  some  instances  in  which  it  is  desirable  to 
assess  subjects’  ability  to  choose  the  true  state  of  the 
world  without  taking  into  account  the  size  of  their 
judgments  of  confidence.  When  subjects  attempt  to 
maximize  the  size  of  reported  i//(Ht|D),  measures  of 
their  performance  based  upon  aggregations  of  i//(Ht|D) 
can  be  misleading  (Schum,  Goldstein,  Southard,  1966). 
The  dichotomous  measure  provides  additional  informa¬ 
tion  which  is  often  useful  in  evaluating  subjects’  in¬ 
ference  behavior. 

In  evaluating  inference  accuracy  we  have  thus  made 
free  use  of  our  ability  to  specify  Ht.  Since  the  laws  of 
probability  apply  in  our  research  as  they  do  elsewhere, 


Hypotheses: 
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we  suffer  when  results  of  the  best  aggregate  of  the  evi¬ 
dence  do  not  coincide  with  our  knowledge  of  Ht.  How¬ 
ever,  this  happens  only  infrequently.  In  some  of  our 
current  experiments  we  have  found  it  necessary  to  dis¬ 
card  scenarios  for  which  Bayesian  P(H,|D)  is  not  the 
largest  value  in  the  calculated  terminal  posterior  prob¬ 
ability  distribution.  In  entirely  abstract  or  in  quasi- 
realistic  experimental  situations  such  as  ours,  it  is  diffi¬ 
cult  to  see  how  there  could  be  accuracy  criteria  other 
than  those  involving  either  experimenter-known  Ht  or 
the  hypothesis  emerging  as  most  probable  from  Bayes* 
theorem  calculations. 

Speed,  in  addition  to  accuracy,  is  a  criterion.  In  a 
diagnostic  systems  context  one  experimental  question 
relates  to  the  speed  at  which  competing  alternative 
diagnostic  systems  can  arrive  at  conclusions  from  the 
same  evidence.  Two  competing  diagnostic  systems 
(i.e.,  two  different  procedures  for  producing  output) 
may  arrive  ultimately  at  the  same  conclusion  One  basis 
for  a  choice  between  these  alternative  systems  concerns 
the  speed  with  which  the  conclusion  has  been  reached. 
This  question  is  basic  to  the  major  performance  com¬ 
parison  in  our  systems  experiments.  This  comparison, 
of  course,  is  between  the  PIP  method  for  producing 
P(H|D)  (method  2  above)  and  the  method  involving 
direct  estimation  of  P(H  D)  (method  1  above).  The 
reasons  why  PIP  should  be  superior  have  been  dis¬ 
cussed  earlier  in  this  paper.  Another  research  question 
involving  speed  concerns  the  effects  of  time  stress  on 
the  ability  to  make  estimates  of  P(DH)  and  P(H  D). 
An  experiment  on  this  issue  will  be  described  briefly 
in  the  next  section. 

Some  current  methodological  considerations 

A  simulated  diagnostic  task  environment 

Following  is  a  discussion  of  the  key  methodological 
features  of  our  current  series  of  (three)  experiments. 
The  basic  issue  in  these  experiments  concerns  a  com¬ 
parison  between  posterior  probabilities  produced  by  the 
PIP  method  and  those  estimated  by  subjects  directly 
from  data.  Subjects  in  these  experiments  performed  in 
a  simulated  military*  threat-diagnosis  context.  Only  a 
few  details  of  the  data  base  need  to  be  mentioned.  In  a 
fictitious  “world”  there  were  four  countries:  Aggressor 
I,  Aggressor  II,  a  neutral  country,  and  a  country  called 
US.  There  were  four  states  of  the  world  (hypotheses) 
of  interest:  Aggressor  I  is  planning  to  attack  US;  Ag¬ 
gressor  II  is  planning  to  attack  Aggressor  I,  Aggressor 
II  is  planning  to  attack  the  neutral  country;  and  peace 
will  prevail.  The  hypotheses  in  this  set  were  defined 
by  the  experimenters  to  be  mutually  exclusive  and  ex¬ 
haustive.  Although  there  are  other  logically  possible 
states  of  the  world,  the  subjects  were  instructed  to  dis¬ 
regard  them  as  possible  causes  for  the  evidence  they 


would  observe.  Each  of  the  four  hypotheses  generated 
data  from  as  many  as  18  data  classes.  Each  data  class 
defined  a  class  of  activity  in  the  fictitious  world  such 
as  “proportion  of  strategic  bomber  force  on  alert  status 
in  the  homeland  of  Aggressor  I.”  Further,  each  data 
class  had  five  subclasses  each  representing  a  possible 
state  or  condition  of  the  data  class  in  question.  The  five 
subclasses  in  the  above  example  referred  directly  to 
the  proportions  of  strategic  aircraft  on  alert  10,  30, 
50,  70,  and  90  per  cent). 

Objective  probabilities  of  the  form  P(DjU  H,)  were 
initially  specified  by  the  experimenter  under  each  of  the 
four  hypotheses,  where  j  refers  to  the  jth  data  class,  k 
to  the  kth  subclass  of  data  class  j,  and  i_  to  one  of  the 
four  hypotheses.  In  addition,  joint  conditional  prob¬ 
ability  values  of  the  form  P(DKX,DKy|Hi)  were  defined 
since,  as  discussed  below,  conditional  nonindependence 
among  data  items  was  a  feature  of  the  present  data 
base  (in  the  above  expression  g  and  h  are  two  data 
classes  and  x  and  y  are  subclasses  within  g  and  h). 
These  objective  probabilities  formed  the  model  from 
which  samples  of  data  (scenarios)  were  generated  under 
each  of  the  four  hypotheses.  Each  scenario  contained 
single  subclasses  selected  at  random,  one  each  accord¬ 
ing  to  the  P(DjU|Hi)  or  P(DK*,Dhy|Hi)  distributions 
appropriate  to  the  hypothesis  in  question.  Subclasses 
were  generated  from  as  many  as  18  data  classes.  Sce¬ 
nario  size,  an  experimental  variable  in  two  of  the  experi¬ 
ments,  was  manipulated  by  regulating  the  number  of 
subclasses  generated  in  each  scenario. 

An  IBM  7094-1401  system,  with  interface  provided 
by  four  digital  display  consoles  on  line  to  the  7094, 
was  utilized  in  five  ways:  (a)  to  generate  scenarios 
from  the  objective  probability  matrices  specified  by 
the  experimenters  (the  number  of  scenarios  represent¬ 
ative  of  each  hypothesis  conformed  to  prior  probability 
distributions  also  specified  by  the  experimenters),  (b) 
to  arrange  these  scenarios  in  predetermined  sequences 
according  to  experimental  objectives,  (c)  to  provide  a 
means  for  presenting  items  of  evidence  from  these 
scenarios  to  subjects  at  specified  time  intervals,  (d)  to 
provide  the  means  for  subjects  to  accumulate  and  re¬ 
trieve  the  past-history  information  necessary  in  their 
specification  of  evidence  impact,  and  (e)  to  provide 
the  means  for  automated  collection  and  analysis  of 
subjects’  P(D  H)  and  P(H|D)  responses.*  The  com¬ 
puter  function  allowing  for  past-history  accumulation 
and  retrieval  is  of  some  importance  and  bears  further 
explanation.  Subjects  could  call  up  matrix  presentations 

*Computer  programs  for  scenario  generation  and  presentation, 
past-history  accumulation,  and  response  collection  were  de¬ 
veloped  by  Jack  F.  Southard  and  Mrs.  Louise  Womboll  of 
the  Human  Performance  Center.  Mrs.  Carol  Estep  developed 
programs  necessary  for  statistical  analysis  of  subjects’  responses. 
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on  their  digital  displays.  These  matrices  contained  the 
current  relative  frequency  of  each  subclass  of  any 
single  data  class  or  the  current  joint  relative  frequency 
of  subclasses  from  any  pair  of  data  classes  under  each 
of  the  four  hypotheses.  The  7094  was  programmed 
to  update  these  relative  frequencies  as  the  experiment 
progressed.  From  these  matrices  subjects  could  obtain 
estimates  of  conditional  probabilities  having  the  fol¬ 
lowing  forms:  P(Djk|Hi),  P(Dgx,Dhy|Hi) ,  P(Dh>|H>,D*J. 
These  probabilities  were  used  by  subjects  in  their 
estimates  of  evidence  impact.  After  an  experimenter- 
controlled  data-processing  time  limit,  a  subject  entered 
his  response  directly  into  the  digital  display  by  means 
of  a  light  pen. 

Eight  highly  experienced  subjects  served  in  the  three 
experiments.  “Highly  experienced”  means  that  each 
subject  had  served  in  a  number  of  previous  experi¬ 
ments  involving  the  estimation  of  conditional  prob¬ 
abilities  in  complex  circumstances.  In  addition,  each 
subject  was  given  intensive  training  relative  to  his  par¬ 
ticular  task  in  the  current  experiments.  Four  subjects 
served  as  P(DjH)  estimators  in  a  scmi-PIP  condition. 
The  semi-PIP  condition  is  discussed  below  in  detail. 
These  subjects  specified  the  impact  of  various-sized 
subsets  of  data  from  each  scenario.  The  7094  per¬ 
formed  the  aggregation  of  these  impact  statements  ac¬ 
cording  to  Bayes’  theorem.  Each  subject  had  his  own 
console  and  worked  independently  of  the  other  three 
subjects.  In  addition,  all  subjects  were  presented  with 
identical  data.  For  each  scenario,  therefore,  there  were 
four  separate  Bayesian  calculations  of  terminal  P(H|D), 
each  calculated  from  the  impact  specification  of  a 
single  subject.  The  other  four  subjects  each  estimated 
the  quantity  P(H!D)  directly  from  the  scenarios.  That 
is,  they  assessed  evidence  impact  and  performed  all  of 
the  necessary  aggregation  of  this  evidence.  They  also 
responded  directly  into  the  digital  display  consoles  and 
they  observed  the  same  scenarios  as  subjects  in  the 
scmi-PIP  condition. 

A  semi-PIP  condition 

In  a  PIP  system  several  or  many  experts,  each 
knowledgeable  about  a  certain  class  of  evidence,  would 
provide  probabilistic  expressions  describing  the  diag¬ 
nostic  impact  of  evidence  items  as  they  occur.  The 
data-processing  capability  of  such  a  system  will  depend 
upon  both  the  number  of  experts  available  to  evaluate 
the  various  classes  of  evidence  and  upon  the  amount 
of  evidence  each  expert  must  process.  If  the  rate  of 
accumulation  of  certain  classes  of  evidence  is  high, 
P(DIH)  experts  may  not  have  time  to  process  and 
respond  to  each  individual  datum.  Indeed,  they  may 
have  to  process  clusters  or  subsets  of  data  and  provide 
a  single  P(D|H)-type  response  (under  each  hypothesis) 


which  will  specify  the  diagnostic  impact  of  the  entire 
subset.  This  assumes,  of  course,  that  no  data  arc  to  be 
discarded.  The  point  is  that,  on  occasion,  some  evi¬ 
dence  aggregation  may  have  to  be  performed  by  P(D  H) 
experts.  The  PIP  system  notion  cannot  guarantee  that 
the  entire  task  of  evidence  aggregation  will  be  done  by 
computers.  It  docs  guarantee,  however,  that  the  total 
amount  of  necessary  aggregation  could  be  broken  down 
into  smaller  and  more  manageable  aggregations.  The 
success  of  the  PIP  idea  depends  to  a  great  extent  upon 
how  well  men  can  estimate  P(D  H)  or  some  related 
quantity.  The  difficulty  of  this  estimation  task  depends 
at  least  upon  the  availability  of  historical  records,  upon 
processing  time,  upon  how  equivocal  and  conflicting 
the  data  arc,  upon  the  size  of  the  evidence  set  being 
evaluated,  and  upon  whether  or  not  there  arc  nonin- 
dcpcndcncics  among  data.  It  is  of  interest,  therefore,  to 
evaluate  P(DH)  [and  P(H[D)]  estimation  by  men 
under  conditions  which  simulate  some  of  these  prob¬ 
lems. 

In  the  current  scries  of  experiments  subjects  in  the 
PIP  group  specified  the  diagnostic  impact  of  clusters 
or  subsets  of  data.  The  size  of  the  clusters  or  subsets  of 
data  was  a  variable  in  one  of  the  experiments.  A  single 
response  was  provided  under  each  hypothesis  which 
specified  the  aggregate  impact  of  a  data  subset.  The 
task  was  made  quite  complex  because  conditional  non- 
indcpendcncics  existed  between  data  within  a  subset 
and  between  data  from  one  subset  and  data  from  some 
preceding  subset.  In  order  to  prescribe  the  aggregate 
impact  of  a  subset  of  data,  a  PIP  subject  frequently 
had  to  aggregate  three  types  of  conditional  probabilities: 
P(DJk|H,),  PCDg^D^lH,),  and  P(DhJ|H,DKX).  In  pre- 
scribing  the  precise  form  which  PIP  subjects’  responses 
would  take,  we  exploited  the  likelihood  principle.  Ac¬ 
cording  to  this  principle,  of  considerable  importance  in 
applications  of  Bayes’  theorem,  P(D[H)  need  only  be 
defined  up  to  a  multiplicative  constant.  In  prescribing 
the  impact  of  a  subset  of  evidence,  a  PIP  subject  as¬ 
signed  a  simple  ratio  across  the  set  of  hypotheses.  The 
number  1  was  assigned  to  the  hypothesis  under  which 
the  occurrence  of  a  subset  of  data  seemed  least  likely. 
Numbers  assigned  under  the  other  three  hypotheses  in¬ 
dicated  the  strength  of  a  subject’s  opinion  regarding 
the  relative  likelihood  of  the  evidence  subset  under  each 
of  these  hypotheses.  Numbers  were  allowed  in  the  range 
1  to  999999.  An  assumption  which  this  method  makes 
is  that,  if  the  three  possible  independent  likelihood 
ratio  estimates  were  made  by  pairing  each  of  the  three 
hypotheses  with  the  one  judged  least  likely,  the  same 
ratio  as  that  described  above  would  be  recovered. 

There  were  always  three  and  as  many  as  six  subsets 
of  evidence  presented  in  each  scenario.  The  computer 
system  aggregated  the  impact  specifications  made  by 
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PIP  group  subjects  for  eaeh  data  subset  within  a 
scenario.  The  resulting  P(H  D)  distributions  were 
never  displayed  to  them.  Subjects  who  estimated 
P(H|D)  were  required  to  make  the  same  complex 
impaet  assessments  but  they  were  also  required  to 
perform  the  other  aggregation  neeessary  for  opinion 
revision.  Their  response  in  two  of  the  three  experi¬ 
ments  was  in  the  form  of  a  normalized  P(H|D)  dis¬ 
tribution.  In  the  third  experiment  the  subjects  responded 
by  assigning  a  ratio,  similar  to  that  described  above, 
whieh  indicated  posterior  opinion  about  the  truth  of 
hypotheses. 

Conditional  nonindependence  of  evidence 

Part  of  the  task  of  evidence  evaluation  consists  of 
appraising  the  influence  various  evidence  items  have 
upon  one  another.  The  diagnostic  impaet  of  some  cur¬ 
rent  event  may  be  considerably  different  if  it  is  known 
that  some  other  event  has  occurred.  Two  events  are 
said  to  be  nonindependent  if  the  oeeurrenee  of  one 
event  somehow  influences  the  probability  of  oeeurrenee 
of  some  other  event.  In  a  diagnostic  context,  sueh 
nonindependenee  may  be  implied  or  mediated  by  the 
truth  of  some  hypothesis  being  considered.  That  is, 
the  underlying  proeess  eausing  the  nonindependenee 
is  represented  by  the  truth  of  some  hypothesis  being 
considered.  Nonindependenee  mediated  by  the  truth 
of  some  hypothesis  is  called  conditional  nonindepen¬ 
dence,  Another  elass  of  event  nonindependenee  in¬ 
cludes  instances  in  whieh  the  probabilistic  relationship 
between  events  is  implied  or  mediated  by  some  proeess 
not  represented  by  the  truth  of  any  hypothesis  being 
considered.  Further  diseussions  of  these  two  elasses  of 
nonindependenee  ineluding  examples  of  eaeh  type  are 
to  be  found  in  reports  by  Edwards  (1963)  and  Schum 
(1966b). 

Formally,  two  items  of  evidence  (D,,  D2)  are  in¬ 
dependent  in  the  light  of  some  hypothesis  (H,)  if: 

P(Dj|Hj,  D,  =  P(Dj|Hi)  (6) 

Equation  6  expresses  the  condition  in  whieh  the  prob¬ 
ability  of  D2i  when  the  truth  of  H>  is  assumed,  is  the 
same  whether  or  not  D,  has  been  observed. 

Equation  6  implies  that: 

P(D„  D,|H,)  =  P(DI|HI)  P(D,!H,)  (7) 

If,  however,  P(D,|H,,  D.)  ^  P(D2;Hj)  for  some  H,, 
then  D,  and  D2  are  said  to  be  nonindependent  when  Fh 
is  true,  i.e.,  there  is  nonindependenee  between  Dj  and 
D2  conditional  upon  the  truth  of  Hi. 

Conditional  nonindependenee  between  eertain  data 
elasses  was  incorporated  in  the  data-generation  models 
of  all  of  our  eurrent  experiments.  The  aetual  procedure 
for  instituting  conditional  nonindependenee  in  data- 
generation  models  is  discussed  elsewhere  (Sehum, 
1965b).  In  the  present  experiments  nonindependenee 


existed  only  among  pairs  of  data  elasses,  i.e.,  there 
were  no  higher-order  nonindependeneies.  The  reason 
for  incorporating  this  feature  is  that  there  is  often 
additional  diagnostieity  in  conditionally  nonindependent 
evidence.  Recognition  and  exploitation  of  conditional 
nonindependenee  ean  allow  one  to  move  more  quiekly 
toward  the  aeeeptanee  of  some  hypothesis.  In  other 
eases,  independent  consideration  of  two  or  more  data 
may  lead  toward  aeeeptanee  of  one  hypothesis  while 
consideration  of  suspected  conditional  nonindependenee 
of  these  data  may  lead  toward  aeeeptanee  of  some 
other  hypothesis.  In  any  ease,  it  is  of  interest  to  in¬ 
vestigate  how  well  men  ean  reeognize  and  exploit  con¬ 
ditional  nonindependeneies  in  the  task  of  specifying 
evidence  impaet. 

Figure  1  illustrates  the  formal  requirements  of  an 
impaet-speeifieation  task  when  (a)  samples  of  evidence 
of  various  sizes  arrive,  (b)  some  of  the  evidence  items 
are  nonindependent,  and  (e)  when  a  single  impaet- 
speeifying  response  is  required  for  eaeh  sample  as  it  ar¬ 
rives.  In  the  example,  four  samples  of  evidence  have 
come  into  existence  at  successive  times,  T,,  T,,  T3,  and 
T,.  The  samples  at  T,  and  Tt  contain  two  items  eaeh, 
the  sample  at  T3  contains  three  items,  and  the  sample 
at  T4  contains  four  items.  A  single  response,  sueh 
as  a  probability  of  the  form  P(D[H)  assigned  under 
eaeh  hypothesis  or  a  simple  ratio  like  that  described 
above,  is  required  for  eaeh  sample.  At  T3,  for  example, 
this  response  must  characterize  the  aggregate  impaet 
of  D5,  D6,  Dt.  Conditional  nonindependenee  among 
data  items  is  indicated  in  Figure  1  by  connecting  lines 
sueh  as  those  between  D,  and  D3  or  between  D5  and 
D6.  Nonindependenee  of  items  within  a  sample  is  il¬ 
lustrated  by  the  pairs  D5,D*  and  D«,Df.  This  means,  for 
example,  that  the  joint  impaet  of  D5  and  Dc,  when  one 
of  the  hypotheses  is  true,  is  different  from  the  total 
impaet  arising  from  independent  consideration  of  these 
data,  i.e.,  that  P(D„Dt|H)  ^  P(D*|H4)  P(D6|,)  for 
some  Ht.  Nonindependenee  between  an  item  from  one 
sample  and  an  item  from  some  preceding  sample  is 
illustrated  by  the  pairs  D3,Dt  and  Dn,D<.  This  means, 
for  example,  that  the  impaet  of  D3  has  been  altered  by 
knowledge  that  D,  has  occurred  under  the  assumption 
of  the  truth  of  some  Ht. 

In  Figure  1  there  is  an  equation  for  eaeh  of  the  four 
temporally  separated  evidence  samples.  The  left-hand 
side  of  eaeh  equation  is  an  expression  for  the  probability 
of  the  entire  sample  given  one  of  the  hypotheses.  Sueh 
a  probability  (or  an  analogous  quantity)  is  required 
under  eaeh  hypothesis  being  considered  in  order  to 
speeify  the  aggregate  impaet  of  the  sample  (a  single 
response  means  one  number  under  eaeh  hypothesis). 
The  preeise  form  of  eaeh  of  the  four  left-hand  ex¬ 
pressions  in  Figure  1  depends  upon  the  number  of 
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Figure  1 — Formal  requirements  of  a  sample  diagnostic  impact- 
specification  task  when  evidence  is  conditionally  nonin¬ 
dependent 

items  in  the  sample  and  upon  whether  or  not  there 
is  conditional  nonindcpendcncc  involving  evidence 
from  this  sample.  As  an  example,  consider  the  data  at 
T2.  The  expression  for  the  aggregate  impact  of  these 
data  is  actually  a  probability  of  the  form  P(D3,D4j 
H>,D,,D2).  But,  since  both  D3  and  D4  are  independent 
of  D2  (under  every  hypothesis),  the  above  expression 
can  be  written  as  P(D3,D4  Hi,D, ).  D,  is  included  in 
this  expression  since  D.}  and  D,  are  nonindependent  in 
the  light  of  one  of  the  hypotheses.  Similarly,  P(D,,DP, 
D7|Hi,DI,D2,D3,D4)  =  P(D„Ds,D7  H,)  since,  in  the 
example,  D3,  De,  and  D7  arc  all  completely  independent 
of  D,,  D2,  D3,  and  D,.  The  right-hand  side  of  each 
equation  contains  the  “lower-order”  conditional  prob¬ 
abilities  involved  in  the  specification  of  total  impact 
for  each  sample.  The  exact  form  of  each  of  the  con¬ 
ditional  probabilities  in  the  right-hand  side  of  each 
equation  depends  upon  the  presence  or  absence  of 
conditional  nonindcpendcncc  and  upon  what  has  al¬ 
ready  been  estimated  and  incorporated  into  Bayes' 
theorem.  At  T2,  for  example,  there  is  some  under 
which  the  impact  of  D3  is  changed  because  D,  was 
observed.  Now  the  joint  effect  of  D,,D*  is  specifiable 
by  a  probability  of  the  form:  P(D,,D3|Hi)  = 


P(D3|Hi,Di)  P(Di|H,).  Observe,  however,  that  PfD^H.) 
has  already  been  incorporated  at  T,  (it  is  assumed  in 
the  present  example  that  the  evidence  impact  estimates 
are  being  immediately  aggregated  by  Bayes’  theorem). 
At  T2,  therefore,  the  appropriate  probability  estimate, 
when  D3  is  nonindependent  of  Dt  given  some  Hi,  is  of 
the  form  P(D3|H,,Dt).  Also  observe  at  T*  that  D3 
and  D4  arc  independent  under  every  hypothesis.  The 
right-hand  side  of  the  equation  at  T2  could  be  written 
as  P(D3|H„D,)  P(D4|Hi,D, ),  but  P^H^D,)  = 

P(D4JH,).  The  probabilities  shown  in  Figure  1  are 
those  which  will  reflect  nonindependence  when  it  oc¬ 
curs.  For  those  hypotheses  under  which  data  arc  in¬ 
dependent  the  form  could  be  changed.  Suppose  at 
Ti,  under  some  hypothesis  (Hk),  Dt,  D2,  D3,  and  D4 
are  all  independent.  The  equation  at  T2  would  then 
simply  be  P(D3,D4|Hk)  -  P(D,|Hk)  P(D4|Hk). 

The  example  in  Figure  1  can  now  be  related  to  the 
task  of  the  subjects  who  served  in  the  PIP  group  in  our 
current  experiments.  Their  basic  task  was  to  estimate 
conditional  probabilities  similar  to  the  left-hand  ex¬ 
pressions  in  the  equations  of  Figure  1.  Their  actual 
response,  however,  was  in  simple  ratio  form  (the 
number  1  being  assigned  to  the  hypothesis  under  which 
the  data  sample  was  least  likely,  etc.).  They  had  ac¬ 
cess  to  lower-order  probabilities  similar  to  those  in  the 
right-hand  side  of  the  equations  in  Figure  1.  These 
probabilities  came  from  the  simulated  historical  records 
(relative  frequency  matrices)  which  subjects  could  call 
up  on  their  digital  display  consoles.  PIP  subjects  were, 
therefore,  required  to  aggregate  the  lower-order  con¬ 
ditional  probabilities  in  order  to  formulate  their  ag¬ 
gregate  impact  assessments.  Subjects  were  given  ex¬ 
tensive  training  in  all  but  one  of  the  formal  aspects  of 
this  task.  They  were  never  told  how  to  combine  the 
lower-order  conditional  probabilities.  Their  responses 
were,  in  fact,  complex  judgments  based  upon  mental 
aggregation  of  the  lower-order  conditional  probabilities. 

The  posterior  probabilities  resulting  from  Bayesian 
aggregation  of  the  PIP  subjects’  responses  were  com¬ 
pared,  in  all  experiments,  with  posterior  probabilities 
estimated  directly  from  the  same  evidence  by  four 
other  subjects.  These  four  non-PIP  subjects  had  the 
same  task  of  impact  assessment  as  PIP  subjects  but 
were  required  to  do  the  additional  aggregation  tasks 
required  in  posterior  probability  estimation. 

Some  experimental  variables 

Analysis  of  the  data  from  our  current  scries  of 
three  experiments  is  not  yet  completed  as  this  paper  is 
being  written.  1  can  at  least  mention  the  variables 
which  were  manipulated  in  these  experiments.  In  the 
first  experiment  we  varied  the  rate  of  accumulation  of 
scenario  evidence.  Scenarios  always  contained  six  items. 
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In  one  condition  the  six  items  were  presented  one  at 
a  time.  Subjects  (in  both  groups)  had  three  minutes  to 
evaluate  each  item.  In  the  second  condition  the  six 
items  were  presented  in  successive  subsets  of  three 
items  each.  Subjects  had  three  minutes  to  process  each 
subset.  In  the  third  condition  all  -six  items  were  pre¬ 
sented  simultaneously.  Subjects  had  three  minutes  to 
process  the  entire  scenario.  In  the  second  experiment 
we  held  the  rate  of  accumulation  constant  (at  three  items 
per  three-minute  processing-time  period)  and  varied 
the  size  of  scenarios,  i.e.,  the  total  amount  of  evidence. 
Scenarios  contained  either  6,  12,  or  18  items.  T  he  third 
experiment  was  similar  to  the  second  except  that  we 
attempted  to  unscramble  the  confounding  relationship 
between  scenario  size  and  scenario  diagnosticity.  This 
confounding  problem  was  discussed  in  an  earlier  sec¬ 
tion  of  this  paper.  Briefly,  scenarios  of  increased  size 
become  typically  more  diagnostic  of  Ht.  One  is  never 
sure  whether  the  decrease  in  inference  accuracy  (rela¬ 
tive  to  Bayes’  theorem)  as  amount  of  evidence  in- 
crases  is  due  to  the  increased  amount  or  due  to  the 
increased  diagnosticity  of  the  evidence.  PIP  and  non- 
PIP  subjects’  performance  was  compared  under  nine 
experimental  treatment  combinations  resulting  from 
three  levels  of  scenario  diagnosticity  and  three  levels 
of  scenario  size  (6  items,  12  items,  or  18  itefs). 

REFERENCES 

1  G.  E.  BRIGGS  and  D.  A.  SCHUM  Automated  Bayesian 
hypothesis  selection  in  a  simulated  threat-diagnosis  sys¬ 
tem .  In  J.  Spiegel  and  D.  E.  Walker  (Eds.)  Informalion 
Syslems  Sciences:  Proceedings  of  the  Second  Congress 
Spartan  Books,  Washington,  D.C.  1965 

2  W.  EDWARDS  Dynamic  decision  theory  and  proba¬ 
bilistic  information  processing.  Hum.  Factors  4,  59-73 
1962 

3  W.  EDWARDS  Probabilistic  information  processing  in 
command  and  control  systems.  University  of  Michigan, 
Institute  of  Science  and  Technology  Ann  Arbor,  Michi¬ 
gan  ESD  Tech.  Docu.  Rep.  No.  62-345  February  1963 

4  W.  EDWARDS  Non-conservative  probabilistic  informa¬ 
tion  processing  systems.  University  of  Michigan,  Insti¬ 
tute  of  Science  and  Technology  Ann  Arbor,  Michigan 
Contract  AF  19(628-2823,  Final  Report  1966 


5  R.  J.  KAPLAN  and  J.  R.  NEWMAN  Studies  in  prob¬ 

abilistic  information  processing.  IEEE  Transcations 
on  Human  Factors  in  Electronics  HFE-7,  49-63  1966 

6  C.  R.  PETERSON,  R.  J.  SCHNEIDER  and  A.  J.  MIL¬ 

LER  Sample  size  and  the  revision  of  subjective  prob¬ 
abilities.  .J  Exp.  Psychol.  69,  522-527  1965 

7  J.  F.  SOUTHARD,  D.  A.  SCHUM,  and  G.  E.  BRIGGS 
An  application  of  Bayes'  theorem  as  a  hypothesis-selec¬ 
tion  aid  in  a  complex  information-processing  system. 
Aerospace  Medical  Research  Laboratories  Wrighl- 
Palterson  Air  Force  Base,  Ohio  AMRL  Tech.  Docu. 
Rep.  No.  64-51  August  1964(a) 

8  J.  F.  SOUTHARD,  D.  A.  SCHUM  and  G.  F.  BRIGGS 
Subject  control  over  a  Bayesian  hypothesis-selection  aid 
in  a  complex  information-processing  system.  Aerospace 
Medical  Research  Laboratories  Wright-Palterson  Air 
Force  Base,  Ohio  AMR1  Tech.  Docu.  Rep.  64-95 
September  1964(b) 

9  D.  A.  SCHUM  Prior  uncertainty  and  amount  of  diag¬ 
nostic  evidence  as  variables  in  a  probabilistic  inference 
task.  Organ  Behav.  Hum.  Perform.  Vol.  I,  No.  1, 
August  1966(a) 

10  D.  A.  SCHUM  Inferences  on  the  basis  of  conditional¬ 

ly  nonindependent  data.  J.  Fxp.  Psychol,  in  press 
(b)  1966  An  expanded  version  is  available  as  AMRL- 

TDR-65-1 61  December  1965 

11  D.  A.  SCHUM,  I.  L.  GOLDSTEIN,  and  J.  F.  SOU  IH 

ARD  The  influence  of  experience  and  information 
fidelity  upon  posterior  probabilities  estimation  in  a 
simulated  threat -diagnosis  system.  Aerospace  Medical 
Research  Laboratories  Wright-Pataterson  Air  Force 
Base,  Ohio  AMRL  Tech.  Rep.  No.  65-25  (a)  April 

1965 

12  D.  A.  SCHUM,  I  L.  GOLDSTEIN  and  J.  F.  SOUTH¬ 

ARD  Further  investigation  of  the  effects  of  reduced 
input  data  fidelity  upon  the  determination  of  posterior 
probabilities  in  a  simulated  threat-diagnosis  system. 
Aerospace  Medical  Research  Laboratories  WrighLPal- 
terson  Air  Force  Base,  Ohio  AMRL  Tech.  Rep.  No. 
65-233  (b)  December  1965 

13  D  A  SCHUM,  I.  L.  GOLDSTEIN,  and  J.  F.  SOUTH¬ 

ARD  Research  on  a  si  undated  Bayesian  information - 
processing  system.  IEEE  Transactions  on  Human  Fac¬ 
tors  in  Electronics  HFE-7  37-48  1966 

14  D  A.  SCHUM,  1  L.  GOLDSTEIN,  W.  C.  HOWELL, 
and  J.  F.  SOUTHARD  Subjective  probability  revisions 
under  several  cost-payoff  arrangements.  Organ.  Behav. 
Hum.  Perform,  in  press  1966 


The  AESOP  testbed:  test  series  1/ 2* 


by  Joseph  M.  Doughty 

The  MITRE  Corporation 
Bedford,  Massachusetts 


INTRODUCTION 

This  paper  describes  a  test  and  evaluation  of  on-line 
digital  computer  planning  aids  having  possible  appli¬ 
cation  to  the  planning  of  tactical  air  combat  opera¬ 
tions.  The  development  and  evaluation  of  these  on¬ 
line  aids  was  accomplished  by  simulating  the  planning 
activities  characteristic  of  the  Fighter  Section  of  a 
Current  Plans  Division  of  a  Tactical  Air  Control 
Center.  The  simulation,  currently  in  operation  in  the 
laboratory,  is  exercised  in  two  ways  — manually  and 
with  computer  aids.  Both  are  carried  out  in  the  con¬ 
text  of  a  Joint  Task  Force  operating  in  a  limited  war 
situation. 

The  manual  simulation  is  concerned  with  the  activi¬ 
ties  related  to  the  generation  of  plans  for  the  alloca¬ 
tion  and  scheduling  of  ordnance  and  aircraft  against 
ground  targets.  The  experimental  subjects  (“the 
planners”)  in  this  simulation  operated  in  accordance 
with  established  Joint  Task  Force  doctrine  and  proce¬ 
dures,  but  without  benefit  of  any  type  of  computer 
aid.  This  manual  simulation  supplied  baseline  data 
that  was  used  to  establish  standards  of  performance 
for  the  force  employment  planning  processes  being 
replicated.  This  manual  simulation  will  be  referred  to 
as  the  T ACC-MANUAL  1/2  System. 

The  second  simulation  differed  from  the  first  only 
to  the  extent  that  the  planners  used  on-line  digital 
computer  capabilities  in  the  accomplishment  of  their 
planning  tasks.  These  capabilities  will  be  referred  to 
as  the  TACC-AESOP  1/2  System. 

Testbed  based  design 

The  TACC-AESOP  1/2  System  was  developed  in 
the  ESD/MITRE  System  Design  Laboratory. 

A  development  program**  of  this  type  requires 
some  means  for  determining  directly  and  rapidly  what 
possible  types  of  on-line  planning  aids  are  useful, 
which  are  inept,  and  which  may  be  actually  detract- 

•The  work  reported  on  in  this  document  was  sponsored  by  the 
Electronic  Systems  Division,  Air  Force  Systems  Command,  under 
Contract  AF 19  (628)-5 1 65. 

••For  a  more  complete  description  of  this  program,  see  (Bennett, 
et  al.,  1965). 


ing  from  the  objective  of  fast  and  effective  planning. 
One  must  have  a  model  or  models  of  the  digital 
planning  aids  to  be  developed.  The  AESOPf  digital 
computer  prototype  is  such  a  model.  It  can  be  used 
to  simulate  various  techniques  for  on-line  information 
retrieval,  file  updating,  and  report  generation;  as  well 
as  the  generation  and  utilization  of  logical  and  mathe¬ 
matical  models  in  order  to  investigate  the  utility  of 
different  forms  of  computer  aids  to  planning.  Com¬ 
parative  data  obtained  from  this  simulation  are  used 
to  guide  the  development  and  refinement  of  on-line 
computer  aids  that  show  promise  for  planning  use. 
In  this  regard,  the  digital  model  building  and  execu¬ 
tion  capability  provided  by  the  simulation  is  one  of 
the  prime  tools  of  the  experimenters  in  this  investiga¬ 
tion.  One  must  also  have  a  model  of  the  type  of  organi¬ 
zational  planning  activity  to  which  the  digital  planning 
aid  is  to  be  applied.  In  this  investigation,  this  takes  the 
form  of  a  laboratory  replica  of  the  Fighter  Section, 
Current  Plans  Division,  of  a  Tactical  Air  Control 
Center.  In  addition,  one  must  have  some  evaluative 
model  through  which  to  obtain  a  better  understand¬ 
ing  of  how  the  aids  might  and  will  be  used,  where  they 
can  help,  and  how  their  design  should  evolve.  This 
evaluative  model  is  discussed  in  the  section  on 
evaluation  procedure . 

AESOP  design  guidance  and  verification  concepts 
are  based  to  a  considerable  extent  on  a  testbed  philos¬ 
ophy.  The  physical  testbed  in  this  instance  is  a  labora¬ 
tory  facility  that  utilizes  an  IBM  7030  computer, 
with  five  on-line  input/output  stations  each  consist¬ 
ing  of  a  cathode  ray  tube  display,  a  light  pencil,  and 
on-line  typewriter  and  high  speed  printer.  Using  this 
facility,  the  AESOP  prototype  planning  aids  can  be 
embedded  in  replications  of  real  time  planning  opera¬ 
tions  that  are  driven  by  simulated  environments 
characteristic  of  the  operations  in  question.  Thus,  ex¬ 
perimental  applications  of  on-line  digital  aids  to 
planning  can  be  exercised  in  order  to  systematically 
evaluate  their  utility.  Such  a  procedure  helps  to  insure 
the  operational  relevance  and  utility  of  the  proto- 
t  AESOP  -  An  Evolutionary  System  forOn-line  Planning. 
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types.  Successful  development  and  test  then  lead 
directly  to  field  application  and/or  to  the  establish¬ 
ment  of  specific  new  requirements  for  computer  as¬ 
sistance. 

As  in  most  system  testbeds,  the  real  time  operations 
and  their  environment  are  only  selectively  repre¬ 
sented  in  the  AESOP  testbed.  That  is,  the  replicated 
planning  activity  is  only  a  part  of  the  total  operation, 
the  rest  being  simulated  in  some  abstracted  form.  The 
outside  environment  which  drives  the  operation  is 
also  simulated  in  abstracted  form.  The  criteria  guid¬ 
ing  the  structuring  of  the  testbed  replica  are  credibility 
and  control.  The  credibility  criterion  is,  to  a  consider¬ 
able  degree,  judgmental;  the  judgment  rests  on  the 
basis  of  an  analysis  of  the  real  world  process  and  a 
validation  of  the  ensuing  model,  with  its  abstractions, 
by  experts,  preferably,  the  real  world  practitioner. 
Control  refers  to  test  control,  the  requirement  that 
critical  test  conditions  are  known  and  controlled  at 
all  times.  The  testbed  must  allow  the  important 
variables  affecting  the  process  to  be  represented  in 
precisely  identifiable  form;  over  a  range  of  values 
within  which  the  process  may  be  stressed. 

The  testbed 

This  section  describes  the  command  structure,  en¬ 
vironmental  interface,  and  the  Fighter  Section  of 
the  TACC  Current  Plans  Division  being  simulated 
in  the  laboratory.  An  appropriate  scenario  provides 
historical  events,  geographical  settin?,  and  associated 
environmental  events  as  a  vehicle  for  the  external 
system. 


The  external  system:  Command  structure 
and  information  flow 

Figure  1  indicates  the  command  structure  which  de¬ 
fines  the  external  system.  The  internal  system  (Fight¬ 
er  Section)  is  bounded  by  heavy  lines.  Command  ele¬ 
ments,  marked  as  simulated,  interface  with  the  Fighter 
Section,  directly  or  through  higher  command  ele¬ 
ments. 

The  Joint  Task  Force  (JTF)  headquarters  insures 
the  necessary  coordination  of  mission  requirements. 
The  Army  component  of  the  joint  operation 
(COMARFO)  provides  the  TACC  Current  Plans 
Division,  through  the  Direct  Air  Support  Center 
(DASC),  with  requests  for  preplanned  (next  day)  close 
air  support  missions  (CAS).  The  DASC  handles  all 
immediate  (on-call)  close  air  support  mission  requests 
but  levies  requests  for  fighter-bomber  sortie  alloca¬ 
tion  on  the  TACC  Current  Plans  Division. 

The  Air  Force  Commander  (COMAFFOR)  re¬ 
ceives  preplanned  interdiction  and  counterair  mission 
requests  from  JTF  headquarters.  These  requests 
are  input  to  the  Current  Plans  Division  from  the 
Deputy  for  Intelligence  (COMAFFOR),  who  also 
provides  the  Current  Plans  Division  with  situation 
briefings  and  target  intelligence.  The  Deputy  for 
Operations  (COMAFFOR)  provides  the  Current 
Plans  Division  with  weather  information. 

Requests  for  fighter-bomber  escort  for  reconnais¬ 
sance  and  airlift  missions  are  input  to  the  Fighter 
Section  by  the  Airlift  and  Recce  Sections  within  the 
Current  Plans  Division. 
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In  the  Current  Operations  Division,  the  Intelligence 
Duty  Officer  (IDO)  provides  the  Current  Plans  Di¬ 
vision  with  information  on  the  current  ground  situa¬ 
tion.  Aircraft  and  crew  status  is  input  to  the  Fighter 
Section  from  the  Fighter  Duty  Office  (E  DO),  Current 
Operations  Division.  Base  and  ordnance  status  infor¬ 
mation  is  provided  by  the  several  Base  Operations. 

The  basic  output  of  the  Fighter  Section  is  a  Frag¬ 
mentary  Order  reflecting  the  requirements  for  meet¬ 
ing  DASC  on-call  and  preplanned  mission  requests. 
This  Fragmentary  Order  is  sent,  when  appropriate, 
to  the  Airlift  and  Recce  Sections  in  the  Current 
Plans  Division  and  to  the  Fighter  Duty  Officer, 
Recce  Duty  Officer,  and  Troop  Carrier  Duty  Officer 
in  the  Current  Operations  Division.  All  Fragmentary 
Orders  are  first  approved  at  the  daily  planning  confer¬ 
ence  by  the  Current  Plans  Director  before  being  sent 
to  the  various  Squadron  Operations  and/or  to  the 
Fighter  Duty  Officer  in  the  DASC. 

The  internal  system:  Process  overview 

Figure  2  illustrates  the  internal  system  planning 
process  and  its  articulation  with  the  external  system 
and  the  data  base.  At  the  beginning  of  each  shift,  the 
planners  check-in,  receive  current  intelligence  brief¬ 
ings,  review  status  information  and  active  Frag 
Orders,  calculate  the  number  and  types  of  sorties 
available  for  next  day’s  missions,  and  obtain  esti¬ 
mates  of  close  air  support  requests  and  requirements 
for  fighter-bomber  escorts  on  recce  and  airlift  mis¬ 
sions. 

The  Fighter  Section  of  the  Current  Plans  Division 
is  concerned  with  planning  for  three  mission  cate¬ 


gories:  (1)  DASC  on-call  close  air  support  require¬ 
ments,  (2)  preplanned  close  air  support  targets,  and 
(3)  preplanned  interdiction  and  counterair  targets. 
Planning  tasks  in  each  of  these  three  mission  cate¬ 
gories  differ  in  several  ways. 

For  DASC  on-call  planning,  the  task  involves  only 
the  allocation  of  specific  squadrons  to  be  on  ground 
alert  at  the  required  time.  Further,  W/E*  solutions 
have  been  completed  by  the  DASC  and  are  not  done 
by  the  Fighter  Section  for  the  DASC  portion  of  the 
planning  process. 

On  the  other  hand,  allocating  for  preplanned  close 
air  support  sorties  requires  W/E  planning  as  well  as 
scheduling  take-off,  time-over  target  (TOT),  landing, 
turn-around,  and  recycling  times. 

Interdiction  and  counterair  planning  docs  not  ordi¬ 
narily  require  W/E  solutions  but  does  require  schedul¬ 
ing  of  take-off,  TOT,  landing,  turn-around  and  re¬ 
cycling  times.  The  W/E  solutions  for  interdiction  and 
counter-air  missions  are  normally  supplied  to  the 
Fighter  Section  by  the  external  system. 

The  internal  systems:  The  test  systems 

There  follows  in  this  section  a  description  of  the 
Manual  and  Computer-based  versions  of  the  internal 
system  that  were  competitive  systems  in  AESOP 
Test  Series  1/2. 

TACC-MANUAL  //2 

This  system  is  a  one-man  version  of  the  procedures, 
manuals,  working  forms,  wall-mounted  displays,  and 

*W/E  planning  involves  determining  the  number  and  type  of  ord¬ 
nance  required  to  achieve  specified  results  on  various  types  of 
targets. 
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work  environment  of  the  Fighter  Section,  Current 
Plans  Division,  Tactical  Air  Control  Center,  of  a 
Joint  Task  Force  operation.  The  model  is  based  on 
Joint  Task  Force  doctrine  and  procedures. 

In  the  test  situation,  three  of  these  systems  were 
operated  simultaneously  but  independently. 


Figure  3 


Figure  3  shows  the  layout  of  the  room  containing 
the  three  systems  in  operation.  Each  cubicle  contains 
a  working  table  on  which  can  be  seen  files  for  manuals 
and  work  forms,  a  wall-mounted  map,  a  large,  wall- 
mounted  mission  board  and  a  small,  wall-mounted 
status  board. 

The  test  observer  sits  in  the  foreground. 

T ACC  ‘AES  OP  l\2 

This  system  is  an  IBM  7030  Computer-based  ver¬ 
sion  of  the  above  manual  system.  The  system  is  an 
experimental  prototype  and  evolutionary  application 
of  generalized  AESOP*  capabilities  to  a  particular 
Air  Force  tactical  planning  task. 

The  system  is  a  display-oriented,  on-line  system 
which  allows  the  single  operator  to  retrieve,  dis¬ 
play,  print,  change,  and  store  planning  information; 
to  call  up  routines  for  performing  arithmetic  functions 
or  to  execute  routine  data  manipulations;  and  to 
generate  system  outputs  — all  on-line  through  the 
media  of  a  CRT  Display,  on-line  typewriter,  light-pen, 
and  high-speed  printer. 

Figure  4  shows  the  layout  of  the  room  containing 
the  three  identical  versions  of  this  system  that  were 
operated  simultaneously  but  independently  during 
the  test.  The  three  planners  are  shown  seated  at  their 
work-stations  in  front  of  the  display  with  typewriters 
and  high-speed  printer  to  their  right.  The  two  test 
observers  are  shown  standing. 

These  three  TACC-AESOP  1/2  systems  time- 
shared  the  IBM  7030,  with  these  systems  having 


Figure  4 


priority  as  foreground  jobs  over  the  background  batch 
processing. 

System  procedures 

There  follows  a  brief,  simplified  description  of 
the  procedures  followed  in  each  of  the  systems  dur¬ 
ing  the  test  sessions.  These  are  discussed  in  the  three 
principal  steps  of  the  planning  process  common  to 
both  systems. 

Step  / :  Initialization.  The  planning  task  began  with 
the  input  into  both  systems  of  (1)  the  set  of  targets 
against  which  they  were  to  plan  missions  and  (2)  the 
resources  available  for  this  purpose.  Since,  during 
the  test  sessions,  this  information  was  preloaded  into 
each  system,  the  planners  simply  checked  it  for  com¬ 
pleteness  and  accuracy,  and  then  proceeded  to  Initia¬ 
lize  their  systems.  Initialization  consisted  of  calculat¬ 
ing  and  storing  basic  planning  information  such  as 
(1)  the  number  of  aircraft  sorties  available  by  squad¬ 
rons,  by  aircraft  type,  and  by  mission  type;  (2)  the 
number  and  types  of  ordnance  required  to  achieve 
the  level  of  target  destruction  requested  for  each 
target  element,  and  (3)  the  distance  and  time  to  each 
target  from  each  of  the  squadron  bases. 

MANUAL  planners  used  tables  and  nomograms 
to  calculate  ordnance  requirements,  and  measured 
distances  and  time  to  targets  on  the  wall-mounted 
maps.  This  information  was  then  copied  onto  a  work 
form  (the  Plan  Generation  Worksheet)  which,  when 
completed,  consisted  of  many  pages  containing  the 
basic  planning  information  needed  to  work  out  an 
allocation  of  resources  to  missions  (the  next  step  of 
planning,  to  be  discussed  below). 

AESOP  planners  used  computational  routines  to 
compute  ordnance  requirements,  distances,  and  times. 
For  example,  the  planner  simply  typed  the  input  mes¬ 
sage  WEAPONS  (  )  and  the  program  computed  the 
number  and  type  of  ordnance  for  each  target  element 
and  the  number  of  sorties  of  each  aircraft  type  needed 
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to  carry  this  ordnance.  This  information  was  auto¬ 
matically  stored  in  appropriate  files  for  later  retrieval 
by  display  or  hard-copy  print.  Similarly,  by  typing 
the  input  message  DISTANCE(),  distance  and  fly¬ 
ing  time  to  each  target  was  computed  and  stored.  As 
a  final  step  in  this  initialization  the  planner  executed 
a  routine  which  copied  into  a  so-called  notebook  file 
selected  information  from  the  prior  computations 
which  allowed  him  to  work  out  a  gross  allocation  of 
resources  in  the  next  step  of  planning 

Step  2:  Gross  planning  and  assignment.  For  both 
systems  this  step  was  the  time  consuming  and  diffi¬ 
cult  part  of  the  process.  The  planner  had  the  task  of 
assigning  and  scheduling  his  available  resources  to 
missions  such  that  the  following  criteria  (listed  in 
order  of  importance)  were  met  as  closely  as  possible 
for  all  targets: 

•  The  requested  level  of  damage  (DP*). 

•  The  requested  time  over  target  (TOT). 

•  The  minimum  use  of  recycled  aircraft  (i.e.,  sortie 
rate,  SR). 

•  The  minimum  total  flying  time  (FT). 

MANUAL  planners  used  the  Plan  Generation 
Worksheet  to  develop  a  gross  plan  first.  This  gross 
plan,  when  completed,  was  a  solution  to  the  problem 
of  resource  allocation  but  lacked  the  details  of  each 
mission;  e.g.,  take-off  time,  time  over  target,  time  of 
return,  next  ready  time  of  recycled  aircraft,  flight  call- 
sign.  These  details  were  next  calculated  and  entered 
on  another  form,  the  A  ire  raft -Target  Worksheet.  With 
this,  the  assignment  of  resources  to  missions  was  com¬ 
pleted. 

ARSOP  planners  were  provided  with  no  computer¬ 
ized  decision  aids  to  this  allocation  problem.  At  the 
stage  of  evolution  of  the  system  at  the  time  of  testing, 
such  aids  as  planning  algorithms  or  Linear  Pro¬ 
gramming  techniques  had  not  been  adapted  for  use 
in  this  TACC  planning  task,  although  work  was  pro¬ 
ceeding  in  this  direction.  Consequently,  the  planners' 
task  in  the  allocation  of  resources  was  basically  the 
same  for  both  systems,  although  differing  in  detail. 
The  AESOP  system  did,  however,  provide  such  aids 
as  the  automatic  generation  of  the  mission  details 
(take-off  time,  time  over  target,  Pk,  time  of  return 
etc.),  automated  bookkeeping  on  remaining  resources 
(number  and  time  of  availability  of  aircraft  and  sorties 
by  squadrons,  aircraft  type,  and  mission  type),  and 
automated  posting  of  all  mission  information  in  a 
central  file  called  MISSION.  The  planners'  decisions 
regarding  the  assignment  of  resources  to  missions 
were  entered  into  the  system  through  a  typewriter 
input  message,  e.g., 

ASSIGN  (  TT  15  27TFS  3  ) 
which  meant,  assign  to  target  number  T 1 1 5  three  air¬ 


craft  from  the  27th  Tactical  Fighter  Squadron.  When 
this  instruction  had  been  processed,  the  assignment 
was  stored  and  displayed  in  the  MISSION  file  to¬ 
gether  with  all  the  automatically  computed  mission 
details.  This  also  updated  those  bookkeeping  files  in 
which  running  accounts  of  resource  utilization  and 
remaining  availability  were  maintained.  Changes  in 
decisions  regarding  assignment  could  be  implemented 
by  first  unassigning,  e.g.,  typing  UN  ASSIGN  (  T 1 1 5 
27TFS  ),  and  reassigning. 

Step  3:  Frag  order  and  mission  posting.  The  out¬ 
put  of  the  two  systems  was  the  Frag  Order  reflecting 
the  plan  developed  in  Step  2.  The  Frag  Order  con¬ 
tained  all  the  information  necessary  for  the  operat¬ 
ing  squadrons  to  carry  out  the  mission  planned  by 
the  Fighter  Section.  In  addition,  the  plan  was  posted 
for  briefing  and  monitoring  purposes. 

In  the  MANUAL  system,  the  Frag  Order  was 
written  by  preparing  a  page  for  each  mission  assigned 
to  each  squadron.  This  involved  copying  into  a  form 
all  the  details  on  each  mission  worked  out  in  the  pre¬ 
ceding  steps  and  adding  call-sign  information.  The 
posting  of  the  mission  was  done  by  copying  from  the 
Flag  Order  onto  a  wall-mounted  mission  board  the 
details  of  each  mission. 

In  the  AESOP  system,  the  Frag  Order  was  printed 
on-line  by  simply  typing  for  each  mission  the  input 
message  DO  A  FRAG  and  adding  the  target  number, 
the  page  number,  and  the  assigned  call-sign.  Mission 
posting  was  automatically  completed  with  the  Frag 
Order  generation  since  all  mission  information  was 
stored  in  the  MISSION  file  where  it  was  available 
for  on-line  display  or  printing,  locally  or  remotely, 
for  briefing  and  monitoring  purposes. 

Evaluation  procedure 

The  evaluative  model  used  in  this  test  asserts  that 
man-machine  systems  should  be  evaluated  on  four 
aspects:  (1)  Formality,  (2)  Utility,  (3)  Excellence, 
(4)  Desirability.  This  classification  scheme  follows 
Jacobs  (1965).  Evaluations  of  systems  on  these  four 
criteria  should  answer  the  following  questions  about 
the  system: 

•  Formality  —Does  the  system  meet  the  require¬ 

ments  and  expectations  of  knowl¬ 
edgeable  TACC  planning  personnel 
insofar  as  credibility  is  concerned? 

•  Utility  —Does  the  system  do  the  job  it  is 

supposed  to? 

•  Excellence  —How  well  does  the  system  ac¬ 

complish  its  goal? 

•  Desirability  —  How  efficient  is  the  system  in  utiliz¬ 

ing  resources  to  accomplish  its 
goal? 
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The  criteria  outlined  in  the  succeeding  section  deal 
primarily  with  the  excellence  and  desirability  of  the 
output  and  of  the  process  in  producing  an  output. 
The  evaluation  procedures,  however,  should  assure 
that  the  formality  and  utility  criteria  are  met  for  the 
sake  of  completeness.  System  formality  was  satis¬ 
fied  prior  to  the  test  series  by  establishing  realistic 
and  noncontradictory  planning  ground  rules  and  pro¬ 
cedures,  reflecting  Air  Force  doctrine  as  published 
and  interpreted  by  Air  University  personnel.  Subjects 
were  trained  according  to  these  specifications,  and 
test  sessions  were  governed  by  these  “formality” 
rules. 

The  utility  criteria  was  met  for  any  plan  output  by 
correcting  it,  after  the  test  session  in  which  it  was 
generated,  so  that  all  errors  (e.g.,  violation  of  re¬ 
source  availabilities)  were  removed.  The  correcting 
was  done  by  deleting  from  the  final  plan  those  parts 
which  were  in  error.  In  other  words,  the  output  that 
was  scored  via  the  method  outlined  in  the  next  section 
was  a  utile  output.  Obviously,  the  extent  to  which 
the  original  plan  did  not  satisfy  the  utility  require¬ 
ments  was  reflected  in  lower  excellence  and  desira¬ 
bility  evaluations,  since  the  erroneous  parts  of  the 
plan  made  no  contribution  whatsoever  to  the  score. 

The  excellence  of  output,  or  how  well  the  plan  satis¬ 
fied  the  goal  of  potential  target  destruction,  was 
measured  as  a  function  of  the  Pk  (probability  of  target 
destruction),  and  TOT  (time  over  target)  measures. 
The  desirability  of  output,  or  the  efficiency  of  com¬ 
mitting  resources  to  the  planning  goal,  was  measured 
as  a  function  of  aircraft  utilization  as  reflected  in  the 
Sortie  Rate  and  Flying  Time  measures. 

Process  excellence,  or  how  well  the  system  oper¬ 
ated  in  producing  its  output,  was  stated  in  terms  of 
operator  opinion  and  attitude.  Process  desirability, 
or  how  efficiently  the  system  utilized  resources  in 
producing  the  output,  was  measured  in  terms  of  total 
time  to  plan. 

Details  on  these  criteria  and  measures  are  given  in 
the  following  paragraphs. 

Criteria  and  measures  of  excellence 
and  desirability 

In  the  following  discussion  plan  output  and  planning 
process  are  distinguished  as  separate  categories  for 
evaluation  purposes. 

Output 

The  approach  was  to  combine  output  measures  of 
excellence  and  desirability  in  such  a  way  that  the  sys¬ 
tems  could  be  discriminated  as  better  or  worse  ac¬ 
cording  to  a  single  criterion  score.  The  following  func¬ 
tion  yielded  such  a  score: 

W  =10*  SPk  4-  8  •  Stot  5  •  Ssr  4  * 
where  W  is  the  overall  worth  score  of  a  plan  and 


S Pk,  S T0Ty  S SR,  and  SFT  are  the  scores  from  the  output 
measures  of  Pk ,  TOT,  Sortie  Rate,  and  Flying  Time, 
respectively,  computed  according  to  the  specification 
given  below.  The  task  of  the  planner  was  to  maxi¬ 
mize  the  above  criterion  function. 

Obviously,  the  worth  function  can  be  modified  if 
more  realistic  weights  are  found  — or  its  entire  form 
can  be  changed.  At  present,  though,  it  is  a  reasonable 
first  pass  at  a  credible  criterion  function,  based  as  it 
is  on  the  opinion  of  people  experienced  in  this  kind  of 
planning. 

Following  are  detailed  discussions  of  the  scoring 
of  the  above  four  key  output  variables  which  subjects 
controlled  and  traded  off  in  generating  a  TACC  plan. 

Planned  P^(P0  levels  for  each  target.  The  penalty 
for  not  meeting  a  desired  P*.  is  a  function  of  both  the 
priority-precedence  and  the  planned  Pk.  The  farther 
a  planned  Pk  falls  below  a  desired  Pk  for  any  target, 
the  lower  the  Pk  score  for  that  target.  The  relative 
penalty  increased  in  order  of  priority-precedence; 
i.e.,  the  higher  the  priority-precedence,  the  higher  the 
penalty  for  Pk  degradation.  Except  for  preplanned 
CAS  targets,  exceeding  the  desired  Pk  does  not  in¬ 
crease  the  criterion  score.  Increasing  CAS  P^’s  be¬ 
yond  the  desired  P*.  will  increase  the  score  only  if 
the  desired  Pk's  on  all  CAS  targets  have  been  met. 
Pk  should  only  be  degraded  after  lower  order  criteria 
have  been  degraded. 

Let  N  =  number  of  targets  in  a  problem  package, 
and  R  =  the  rank  of  a  target  in  the  package,  l^R^N. 
Ranking  will  be  done  on  the  basis  of  priority-prec¬ 
edence. 


Let  Q  = 


*  AT  +  1 
N  +  i 


(I) 


(a)  For  each  target  ^where  desired  P*.  (DPk)  is  less 
than  or  equal  to  P*,  score 


100 

Q 


(2) 


(b)  For  each  target  where  Pk  is  less  than  DP*., 
score 


50 

Q  * 


(3) 


(c)  For  CAS  targets,  jn  the  situation  where  all 
CAS  targets  have  PA. —  DPA.,  and  no  CAS  tar¬ 
gets  have  been  delayed,  score 

_io  ryv+n  pk_  dp,  (4) 

Q  |w  +  KJ  DPk 


as  a  bonus. 
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These  scores  are  designed  to  give  generally  higher 
value  to  higher  ranking  targets,  while  assuring  that 
the  value  of  meeting  the  DPA.  on  any  target  is  more 
than  the  value  of  not  meeting  the  DPA-  on  any  other 
target  regardless  of  rank.  Furthermore,  no  improve¬ 
ment  in  score  is  achieved  by  planning  a  Pk  greater 
than  DPA,  except  in  the  case  of  CAS  targets,  where 
any  available  apportioned  sorties  remaining  after 
all  DP*’s  are  met  should  not  be  left  unassigned. 

Planned  time-on-target  (TOT)  for  each  target . 
Targets  must  be  hit  within  the  range  of  DTOT  through 
LTOT.  If  the  STOT  (scheduled  time  over  target) 
falls  outside  of  this  range,  the  target  is  in  effect  un¬ 
covered,  and  its  Pk  criterion  score  is  set  to  zero.  How¬ 
ever,  within  this  range,  the  closer  the  STOT  is  to 
the  DTOT,  the  higher  the  criterion  score. 

This  score  is  computed  as  follows: 

(a)  For  each  target  where  TOT  is  equal  to  the  de¬ 
sired  (DTOT),  score where  N  is  the  total 

number  of  targets.  (The  fraction,  0/0,  shall  be 
interpreted  as  I .) 

(b)  For  each  target  where  TOT  is  later  than  DTOT, 
but  before  LTOT  (latest  time  over  target), 
score 

N  1  L  I-TOT-DTOT  J  J 

|N  +  1  1  (5) 

In  +  (N  -  r  +  i)j  y  ’ 

where  N  is  the  total  number  of  targets  and  R 
is  the  rank  of  the  target.  _ 

(c)  For  each  target  where  TOT  falls  outside  the 
DTOT: LTOT  range,  score  0. 

DTOT:  LTOT  range,  score  0. 

These  scores  give  higher  value  to  striking  targets 
closer  to  the  DTOT.  In  the  event  of  mission  seg¬ 
mentation,  the  planned  TOT  for  the  first  strike  will 
be  taken  as  the  planned  TOT  for  the  mission. 

Sortie  rate  of  aircraft  as  used  in  the  plan.  Although 
some  aircraft  may  be  recycled  and  flown  against  more 
than  one  target  on  mission  day,  planners  should  keep 
the  used  sortie  rate  as  low  as  possible.  In  short,  air¬ 
craft  should  be  recycled  only  when  resources  or  ap¬ 
portionments  are  stressed. 

This  score  is  computed  by  multiplying  the  number 
of  aircraft  used  in  the  plan  by  100  and  dividing  by  the 
number  of  sorties. 

Total  flying  time .  When  a  choice  of  aircraft  types 
and/or  squadrons  exists,  assignments  should  be  based 
on  the  minimum  mission  flying  time  associated  with 
the  assignment.  Mission  flying  time  =  (one-way  flying 


time  by  aircraft  type)  multiplied  by  (number  of  air¬ 
craft  type  sorties  assigned). 

To  compute  this  score,  multiply  the  total  number 
of  each  aircraft  type  by  its  sortie  rate,  by  its  combat 
radius  in  nautical  miles  and  divide  by  its  combat 
speed  in  knots.  Add  these  values  for  all  aircraft  types. 
The  result  is  the  maximum  available  flying  time 
(FTmflJ.).  The  used  flying  time  (FTK,rd)  will  be  the  sum 
of  each  sortie  times  its  one-way  flying  time  divided 
by  60.  The  score  is: 

100  x  [ -F^~  FTj^'j  (6) 

All  of  the  above  scores  should  range  roughly  be¬ 
tween  10  and  100  for  an  average  plan  with  fifteen 
target  elements  and  stressed  resources.  Also,  the 
scoring  method  has  been  constructed  to  give  higher 
scores  to  greater  worth  as  determined  by  the  criteria 
outlines. 

Process 

In  order  to  assess  the  excellence  of  the  process  of 
generating  the  system  output,  the  operators  of  the 
systems  in  test  were  asked  to  critique  briefly  the  sys¬ 
tems  after  each  run  and  more  comprehensively  at  the 
end  of  the  test  series.  The  objective  was  to  discover 
opinions  and  attitudes  concerning  experienced  ad¬ 
vantages,  disadvantages,  difficulties,  problems, 
needed  improvements,  confidence  in,  and  the  like, 
in  the  use  of  the  systems  to  achieve  the  goal  of  a 
plan  output. 

Process  desirability  is  best  expressed  in  terms  of 
man-hours  or  man-minutes  spent  in  completing  a 
plan.  Since  one-man  teams  were  used  to  operate  the 
test  systems,  the  measure  of  process  desirability  be¬ 
came  simply  the  number  of  minutes  from  start  of 
planning  to  completion  of  the  plan.  Since  planning 
progressed  through  similar  stages  in  each  system, 
time  measures  for  each  stage  were  also  obtained. 

Test  procedures 

AESOP  Test  Series  1/2  was  conducted  in  the 
ESD/MITRE  System  Design  Laboratory  during 
the  period  October  13  through  November  2,  1966. 
In  all,  10  four-hour  test  sessions  were  held  during 
which  60  plans  were  generated,  30  on  each  system. 

Subjects  and  training 

Preceding  the  test  series,  an  intensive  four  week 
period  of  subject  training  was  conducted  on  the 
manual  system  (TACC-MANUAL  1/2)  and  com¬ 
puter-aided  system  (TACC-AESOP  1/2).  Training 
on  the  latter  was  combined  with  final  system  shake- 
down.  The  six  subjects  were  trained  an  equal  amount 
on  both  systems. 

Employees  of  the  MITRE  Corporation  were  used 
as  test  subjects.  Original  plans  had  been  to  use  Air 
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Force  Officers  as  subjects  but  last  minute  pressures 
to  complete  the  test  as  soon  as  possible  did  not  al¬ 
low  sufficient  time  for  their  selection,  assignment, 
and  training. 

All  subjects  were  trained  to  the  point  of  producing 
acceptable  plans  in  each  system  within  the  four  hour 
time  period  allowed.  However,  at  the  time  the  test 
sessions  were  begun  no  subject  had  reached  his  maxi¬ 
mum  performance  level.  The  test  design,  therefore, 
controlled  for  continuing  gains  in  skill  by  alternat¬ 
ing  the  subjects  on  the  systems  in  successive  test 
sessions. 

Test  sessions 

Test  sessions  were  conducted  three  times  a  week, 
with  the  two  competing  systems  being  operated  simul¬ 
taneously  on  the  same  problem.  Since  multiple  copies 
of  each  system  were  available,  three  TACC-MAN- 
UAL  1/2  and  three  7  ACC-AESOP  1/2  systems  were 
operated  concurrently;  the  manual  systems  in  one 
room  and  the  computer-based  systems  in  another. 

Within  each  room  precautions  were  taken  to  isolate 
the  three  planners.  A  maximum  of  four  hours  were 
permitted  to  complete  the  plan. 

Before  each  test  session  all  subjects  were  briefed 
simultaneously.  This  was  a  simulation  of  an  Intelli¬ 
gence  Briefing  on  the  ground  and  air  situation  in  the 
context  of  which  they  were  planning.  A  10-day 
scenario  of  a  hypothetical  Joint  Task  Force  opera¬ 
tion  had  been  written  to  provide  operational-like 
inputs  into  the  “next  day11  planning  shop.  (Fighter 
Section). 

Both  systems  were  pre-loaded*  with  the  information 
necessary  for  the  planning  task:  targeting  requests  and 
aircraft  resources  available  to  the  planner.  Thus, 
in  the  TACC-MANUAL  1/2  system,  this  information 
had  been  typed  onto  the  work  forms  used  by  the  plan¬ 
ners  and  posted  on  the  appropriate  wall-mounted 
status  board  and  map.  In  the  TACC-AESOP  1/2 
system,  this  information  had  been  inserted  into  the 
planning  files  to  be  used  by  the  operator. 

All  planners  began  work  on  the  same  planning  prob¬ 
lem  at  the  same  time.  Planning  continued  without 
interruption  (except  for  occasional  “downs”  due  to 
computer  or  equipment  failure  on  TACC-AESOP 
1/2)  until  a  plan  had  been  completed,  posted,  and 
frag  order  cut. 

One  test  observer  for  the  manual  system  and  two 
for  the  computer-based  system  recorded  data  and 
insured  that  all  test  procedures  were  correctly  fol¬ 
lowed.  All  system  actions  and  the  time  of  such  actions 
were  automatical^  recorded  for  TACC-AESOP 


*An  earlier  test  was  conducted  to  determine  ihe  time  and  accuracy 
of  this  loading  process. 


1/2.  These  records  were  supplemented  by  records 
of  “waiting  time”  (the  time  between  the  initiation  of 
an  action  and  the  system  response)  made  by  the  plan¬ 
ners  and  observer  using  stop-watches.  In  TACC- 
MANUAL  1/2,  planners  time-stamped  each  work 
form  as  it  was  used.  Such  records  on  the  two  systems 
gave  a  detailed,  time  ordered  record  of  the  entire 
planning  process. 

The  output  of  the  manual  system  was  available  in 
the  completed  work  forms  and  a  photographic  record 
of  the  wall-mounted  mission  board  on  which  the  plan 
was  posted.  The  output  of  the  computer-based  system 
was  recorded  by  printing  the  files  containing  the  com¬ 
pleted  plan  and  the  frag  order. 

Over  the  course  of  the  test  series,  planners  were 
required  to  complete  three  debriefing  forms.  One 
was  completed  by  each  planner  at  the  conclusion  of 
each  test  session.  The  purpose  of  this  form  was  to 
record  the  users’  opinions  on  the  systems’  perfor¬ 
mance  during  each  session  and  to  provide  explanatory 
information  for  the  evaluation  of  specific  aspects 
of  planning  problem  solution.  The  second  debriefing 
form  was  completed  at  the  end  of  the  test  series  and 
was  designed  to  elicit  user  preferences  among  sys¬ 
tems  and  sub-systems  and  the  reasons  for  these  prefer¬ 
ences.  The  third  debriefing  form  was  also  com¬ 
pleted  at  the  end  of  the  test.  It  was  a  planners’  critique 
of  the  TACC-AESOP  1/2  system  in  which  the  planner 
was  encouraged  to  discuss  at  length  the  strengths 
and  weaknesses  of  the  computer-based  system  and 
to  make  specific  design  modification  recommenda¬ 
tions. 

Test  design 

The  six  test  subjects  were  divided  into  two  groups, 
A  and  B.  The  two  groups  were  then  alternated  on 
the  two  systems  from  session  to  session  as  follows: 


him 

Test  Session 

TACC-MANUAL  1/2 

TACC-AESOP  1/2 

Group  A 

Group  B 

2 

Group  B 

Group  A 

3 

Group  A 

Group  B 

4 

Group  B 

Group  A 

5 

Group  A 

Group  B 

6 

Group  B 

Group  A 

7 

Group  A 

Group  B 

8 

Group  B 

Group  A 

9 

Group  A 

Group  B 

10 

Group  B 

Group  A 

The  test  problem  for  each  session 

was  the  same  for 

all  planners. 

Equivalent  but  different  problems  were 

used  throughout  the  test,  each  problem  consisting 

of  1 5  target 

elements  and  with 

available  aircraft 
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sortie  resources  within  ±  2  sorties  of  the  number 
required  to  meet  the  requested  PA.  and  DTOT  (de¬ 
sired  time  over  target). 

Test  results 

Analysis  of  the  test  data  is  currently  in  progress. 
Results  will  be  published  as  soon  as  possible. 

Preliminary  analysis  does  indicate,  however,  a 
superiority  in  performance  of  the  computer-based 
system  (TACC-AESOP  1/2)  over  the  manual  sys¬ 
tem  (TACC-MANUAL  1/2)  in  all  measures  of  the 
evaluative  criteria  of  excellence  and  desirability; 
Total  time  to  prepare  a  plan  (process  desirability); 
plan  quality  (a  combination  of  output  excellence  and 
desirability),  and  planner  preference  (process  ex¬ 
cellence). 


The  extent  and  significance  of  the  differences, 
however,  remain  to  be  determined. 
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I.  INTRODUCTION 

The  purpose  of  this  Memorandum  is  to  review  work 
completed  on  a  value-judgment-based  Tactical  Air 
Command  system.  The  problem  of  interest  is  this:  a 
TAC  commander  must  decide  whether  to  grant  a 
particular  request  for  immediate  close  air  support, 
or  to  deny  it  on  the  grounds  that  he  must  conserve 
resources  to  fulfill  a  possible  later  request  of  greater 
importance.  We  have  assumed  that  resources  (mis¬ 
sions  available  for  close  air  support)  are  seriously 
limited  relative  to  demands  and  that  there  is  a  wide 
range  of  importance  in  the  requests.  Whether  or  not 
these  assumptions  characterize  current  TAC  opera¬ 
tions,  they  do  describe  circumstances  in  which  a 
TAC  command  system  should  be  prepared  to  func¬ 
tion.  Since  our  proposed  system  is  based  on  judged 
values  or  utilities,  we  have  called  it  a  Judged  Utility 
Decision  Generator  (JUDGE). 

This  Memorandum  begins  by  summarizing  the 
aspects  of  the  TAC  control  problem  that  are  relevant 
to  our  proposed  system,  and  the  present  solution  to 
that  problem.  Section  1 1 1  presents  a  set  of  formal  rules 
for  making  mission-dispatching  decisions.  These  rules 
assume  that  numerical  measures  arc  available  for 
evaluating  possible  mission  outcomes.  They  first 
translate  values  of  mission  outcomes  into  values  of 
the  missions  themselves;  a  solution  to  the  problem 
of  predicting  the  effectiveness  of  missions  is  crucial 
to  that  translation.  Next,  the  rules  specify  a  procedure 
for  making  dispatching  decisions  based  on  mission 
values;  the  procedure  responds  to  fluctuations  in 
arrival  rates  of  requests  having  various  values. 

•This  research  is  sponsored  by  the  United  States  Air  Foree  under 
Projeet  RAN D -Contract  No.  AF  49(638)- 1 700- monitored  by 
the  Directorate  of  Operational  Requirements  and  Development 
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USAF  Views  or  conclusions  eontnincd  in  this  Memorandum 
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How  the  numbers  representing  outcome  values 
are  obtained  makes  no  difference  to  the  formal 
rules  for  using  them,  so  long  as  they  have  the  ap¬ 
propriate  cardinal-utility  properties.  But  we  assume 
that  expert  judges  should  ordinarily  supply  such  num¬ 
bers.  Section  IV  presents  a  small  experiment  compar¬ 
ing  numbers  obtained  by  several  judgmental  pro¬ 
cedures;  the  data  indicate  that  one  procedure  is  clearly 
preferable  to  the  others,  and  that  the  numbers  so  ob¬ 
tained  compare  well  with  “correct”  numbers  (avail¬ 
able  in  this  experiment  but  not  in  TAC  control  system 
settings).  Thereafter  we  discuss  a  field  study  which 
incorporated  this  procedure  and  was  the  first  test  of  a 
JUDGE  system.  Used  in  the  study  were  experienced 
TAC  officer  subjects  who  made  value  judgments 
about  mission  outcomes  in  a  verbally-described, 
realistic,  limited-war  setting. 

The  designs  both  of  JUDGE  and  of  procedures  for 
evaluating  and  refining  it  illustrate  some  general 
principles  concerning  the  design  and  evaluation  of 
judgment-based  command  systems.  Since  our  views 
on  this  topic  seem  radical  to  us,  and  since  we  know 
of  no  published  presentation  of  similar  views,  we  con¬ 
clude  this  Memorandum  with  a  rather  extensive, 
abstract  discussion  of  the  problem  and  our  opinions 
about  its  solutions.  Our  main  conclusions  are  that  all 
command  systems  must  be  based  on  explicit  or  im¬ 
plicit  value  judgments,  that  it  is  better  if  these  judg¬ 
ments  are  explicit  and  if  formal  decision-theoretical 
algorithms  are  used  to  translate  them  into  decisions, 
and  that  systems  based  on  such  judgments  and  algo¬ 
rithms  are  self-validating. 

II.  THE  TAC  CONTROL  PROBLEM 
The  Tactical  Air  Command  (TAC)  has  numerous 
operational  missions  to  perform  including  air  defense, 
counter  air,  interdiction,  assault,  airlift,  reconnais¬ 
sance,  and  close  air  support.  Close  air  support  (CAS) 
missions  use  tactical  aircraft  in  direct  and  immediate 
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response  to  requests  for  support  from  ground  forces. 
A  special  agency,  the  Direct  Air  Support  Center 
(DASC),  controls  strike  aircraft  on  CAS  missions. 
The  DASC  controls  aircraft  assigned  to  it  by  the  Frag¬ 
mentary  Operations  Order  (frag  order)  written  each 
day  by  the  command  organization  of  the  Tactical  Air 
Commander.  The  DASC  is  collocated  with  the 
Tactical  Operations  Center  (TOC)  of  the  Army,  to 
facilitate  TAC-Army  coordination. 

Requests  are  made  from  the  field  by  the  Forward 
Air  Controller  (FAC)  over  the  Tactical  Air  Request 
Net.  The  FAC  is  an  Air  Force  fighter  pilot  stationed 
with  the  ground  troops.  He  knows  the  capability  of 
the  aircraft  and  of  the  weapons  they  carry,  and  can 
advise  the  ground  commander  about  feasibilities  of 
missions  or  can  suggest  the  use  of  air  in  cases  where 
the  ground  commander  might  not  otherwise  consider 
it.  The  FAC  transmits  Army  requests  for  CAS  to 
the  DASC;  these  requests  are  approved  or  disap¬ 
proved  by  each  echelon  of  Army  command  between 
the  requester  and  the  TOC.  Each  of  these  inter¬ 
mediate  Army  commands  has  an  Air  Liaison  Officer 
to  help  with  the  approval  decisions. 

The  DASC  has  the  authority  to  communicate  with 
air  bases  and  send  strike  aircraft  on  missions  specified 
by  an  Army-approved  request.  Whether  the  final 
decision  to  fulfill  a  request  is  made  primarily  by  the 
Air  Force  or  by  the  Army  depends  mostly  on  the 
personalities  of  the  individuals  involved.  Doctrine 
says  that  when  the  request  gets  to  the  DASC,  it  has 
become  a  requirement  and  must  be  satisfied  if  it  is 
within  that  organization's  capability.  If  all  requests 
for  CAS  could  be  satisfied  immediately,  the  DASC’s 
main  function  would  be  status  recording  and  com¬ 
munications;  it  would  be  necessary  only  to  send  the 
aircraft  off  and  follow  them  to  make  sure  that  they  go 
to  the  target  and  did  what  they  were  supposed  to  do. 
But  TAC  must  be  prepared  to  function  in  situations 
where  many  more  demands  arc  placed  on  its  resources 
than  can  be  satisfied.  In  that  case,  a  TAC  control 
system  for  CAS  must  consider  every  request  as  jt 
arrives  and  decide  whether  it  will  be  fulfilled  or 
whether  planes  will  be  withheld  for  use  in  response 
to  a  possibly  more  important  future  request. 

This  sketch  of  the  TAC  control  problem  highlights 
two  difficult  tasks  that  a  TAC  control  system  should 
perform.  The  first  is  evaluating  the  relative  im¬ 
portance  of  each  request;  the  second  is  predicting 
the  arrival  of  future,  more  important  requests.  If  all 
requests  are  equal  or  nearly  equal  in  importance, 
then  a  first-come  first-served  policy  will  be  as  good 
as  any  other,  and  no  control  problem  exists.  If  re¬ 
quests  vary  in  importance,  but  nothing  whatever  can 


be  said  about  the  likelihood  of  future,  more  important 
requests,  then  although  first-come  first-served  is  not 
a  good  policy,  no  better  one  seems  feasible.  Note, 
however,  that  the  value  problem  has  logical  priority 
over  the  prediction  problem.  What  must  be  predicted 
is  not  merely  the  arrival  of  future  requests,  but  the 
arrival  of  future  requests  more  valuable  than  the  one 
now  being  considered;  such  a  prediction  is  impossible 
unless  the  worth  of  each  request  can  be  measured 
at  least  roughly. 

In  current  TAC  field  operations,  the  importance  of 
each  request  is,  in  principle,  evaluated  by  means  of 
a  priority  number  assigned  to  the  request  by  the  re¬ 
quester.  Requests  transmitted  to  the  DASC  are  sel¬ 
dom  assigned  less  than  priority  “1”  by  their  origina¬ 
tors.  The  informal  rules  specify  that  you  should  not 
ask  for  air  support  unless  you  really  need  it;  if  you  do 
need  it,  you  want  to  be  sure  that  your  request  is  at¬ 
tended  to.  This  procedure,  of  course,  makes  the  prior¬ 
ity  assignment  system  meaningless. 

A  closely  related  problem  is  the  self-adaptation  of 
demand  for  CAS  to  the  supply  of  airplanes.  Re¬ 
questers  all  listen  to  the  network  on  which  requests 
are  made;  if  no  planes  are  available  at  a  given  time, 
no  requests  are  made.  Apparently,  informal  social 
pressures  discourage  unfulfillable  requests.  This 
phenomenon  would  not  occur,  we  believe,  in  any  en¬ 
vironment  in  which  TAC’s  resources  were  less  over¬ 
whelmingly  abundant  relative  to  the  need  for  them 
than  is  the  case  in  typical  exercises. 

Even  in  the  present  abundant  environment,  this 
self-adaptive  mechanism  has  three  regrettable  fea¬ 
tures.  A  larger  supply  of  requests  would  provide 
more  intelligence  information  to  higher  headquarters. 
It  would  permit  higher  level  Air  Force  and  Army 
commanders,  rather  than  requesters  in  the  field,  to 
evaluate  relative  importance  of  potential  requests. 
And  it  would  reduce  the  chances  that  the  man  whose 
urgent  need  develops  relatively  late  in  the  day, 
after  almost  all  of  the  day’s  missions  have  been  flown 
or  assigned,  will  have  to  do  without  needed  CAS. 
So  the  essence  of  our  proposal  is  that  requests  should 
be  encouraged  and  that  procedures  based  on  value 
judgments  should  be  used  to  determine  which  re¬ 
quests  should  be  fulfilled  and  which  should  not. 

A  plane  and  pilot  may  fly  several  CAS  missions 
in  a  day.  How  many  such  missions  they  can  fly  de¬ 
pends  on  the  nature  of  each  mission  and  on  the  turn¬ 
around  time  for  the  plane,  which  may  vary  from  30 
minutes  to  many  days.  At  present,  the  frag  order 
specifies  that  the  DASC  will  have  a  certain  number 
of  sorties  available  at  the  beginning  of  the  day  and 
additional  sorties  available  at  specified  times  there¬ 
after.  TAC  bases  are  responsible  for  turning  their 
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planes  and  pilots  around  sufficiently  fast  to  provide 
the  missions  specified  in  the  frag  order.  This  rigid 
procedure  does  not  allow  unplanned  variations  in 
the  number  and  nature  of  missions  assigned  early 
in  the  day  to  influence  the  number  of  missions  avail¬ 
able  later  — though  in  practice  the  system  operates 
more  flexibly  than  the  formal  description  implies. 
A  TAG  control  system  should  be  designed  to  consider 
explicitly  how  the  number  and  nature  of  early  mis¬ 
sions,  whether  anticipated  or  not,  influence  later 
availability  of  aircraft  when  DASC  is  deciding  on 
early  mission  assignments. 

III.  THE  LOGICAL  DESIGN  OF  JUDGE 

This  section  presents  a  set  of  formal  rules  that  ac¬ 
cepts  as  inputs  statements  about  the  values  of  various 
possible  mission  outcomes,  past  experience  with 
incidences  of  requests  having  various  values,  and 
plane  availability.  Outputs  are  dispatching  decisions 
and  future  plane  availability.  Empirical  questions 
about  the  availability  of  inputs  is  not  a  concern  here. 
Wc  assume  that  values  will  in  general  be  made  avail¬ 
able  by  exploiting  expert  human  judgment,  that  past 
experience  with  requests  will  be  available,  and  that 
information  of  a  specified  nature  about  plane  avail¬ 
ability  will  be  available.  Section  IV  examines  pro¬ 
cedures  for  obtaining  value  judgments. 

In  the  following  discussion,  we  assume  that  value 
should  be  maximized.  For  us,  this  assumption  has 
somewhat  the  status  of  an  axiom:  the  philosophical 
discussion  at  the  end  of  this  Memorandum  examines 
it  and  its  implications  to  some  extent.  We  also  assume 
that  under  conditions  of  uncertainty,  expected  value 
should  be  maximized.  The  notion  of  expected  value 
maximization  as  a  criterion  of  optimal  risky  decision¬ 
making  has  far  too  long  a  history  to  summarize  here. 
For  an  elementary  discussion,  sec  Edwards  (1954). 
For  situations  in  which  the  so-called  gambler’s  ruin 
problem  does  not  arise,  we  know  of  no  alternative 
rule  for  risky  decision-making  that  deserves  con¬ 
sideration. 

JUDGE’S  inputs  include  values  of  targets  that  might 
be  attacked,  probabilities  that  attacks  will  destroy 
their  targets,  and  number  of  planes  avilable.  To 
make  dispatching  decisions,  it  considers  the  value  of 
sending  one  or  more  planes  and  thus  of  perhaps 
destroying  the  target,  and  the  cost  resulting  from  the 
fact  that  if  the  planes  are  sent  against  this  target,  they 
will  not  be  available  to  send  against  other  targets 
later.  In  order  to  evaluate  this  cost  of  later  unavail¬ 
ability,  JUDGE  must  consider  future  requests  and 
future  plane  availability.  JUDGE’S  outputs  are  fore¬ 
casts  concerning  future  plane  availability  and  dis¬ 
patching  decisions.  The  computations  that  transform 


its  inputs  into  its  outputs  are  complicated.  A  technical 
but  nonsymbolic  summary  of  their  nature  follows. 

To  make  computation  possible,  JUDGE  divides 
time  into  a  sequence  of  discrete  periods  called  hori¬ 
zons.  (It  makes  no  difference  to  the  formalities  how 
long  a  horizon  is;  we  tend  to  think  of  it  as  one  or  two 
hours.)  The  crucial  point  about  horizons  is  that  new 
planes,  or  planes  turned  around  after  earlier  mis¬ 
sions,  become  available  for  use  only  at  the  beginning 
of  each  horizon;  the  supply  of  planes  within  a  horizon 
is  taken  as  fixed.  This  oversimplification  permits  a 
computationally  and  intellectually  convenient  divi¬ 
sion  of  the  problem  into  two  parts:  the  dispatching 
problem  and  the  planning  problem.  The  dispatching 
problem  is  exclusively  concerned  with  the  decisions 
made  within  one  horizon  regarding  aircraft  assign¬ 
ment;  the  planning  problem  is  concerned  with  both 
the  process  of  planning  a  day’s  activities  (important, 
for  example,  for  maintenance  planning  purposes)  and 
the  task  of  supplying  to  the  dispatching  algorithm 
some  numbers  that  tie  together  the  several  horizons 
that  make  up  a  day. 

Within  one  horizon,  the  only  decision  to  be  made  is 
how  many  planes,  if  any,  to  send  in  response  to  each 
request.  Each  request  comes  with  two  kinds  of  in¬ 
formation  attached:  a  value  for  destroying  the  target, 
and  a  number,  or  perhaps  a  function,  that  represents 
the  effectiveness  of  planes  against  the  target.  The 
dispatching  rule  knows  how  many  planes  it  has  avail¬ 
able  for  use  in  the  remainder  of  the  horizon,  and  it 
has  a  basis  for  estimating  the  arrival  rates  of  requests 
with  various  values. 

To  use  this  information,  the  dispatching  rule  cal¬ 
culates  an  expected  gain  or  loss  for  each  possible 
number  of  planes  that  might  be  sent.  The  expected 
gain  of  sending  no  planes  is  zero.  The  expected  gain 
of  sending  one  or  more  depends  on  the  target’s  value, 
the  probability  that  the  planes  being  sent  will  kill  the 
target,  and  the  cost  of  the  lost  opportunity  to  use  those 
planes  against  some  later  target.  Calculating  this  op¬ 
portunity  loss  is  complicated,  and  depends  on  the 
forecasted  arrival  rates  for  requests  of  various  values. 
Among  the  various  available  dispatching  decisions, 
the  one  with  the  highest  expected  value  is  selected. 
Increases  in  mission  value,  in  supply  of  planes,  and 
approach  of  the  end  of  the  horizon  all  increase  the 
willingness  of  the  system  to  dispatch  a  plane. 

The  dispatching  rule  operates  within  a  horizon: 
the  planning  problem  is  concerned  with  the  inter¬ 
relations  among  horizons.  The  output  of  the  planning 
algorithm  is  a  forecast  of  the  number  of  sorties  to  be 
available  in  each  horizon.  This  forecast  permits  the 
dispatching  rule  to  find  out  the  value  of  planes  left 
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over  at  the  end  of  a  horizon.  They  are  obviously 
valuable,  since  they  can  be  used  during  the  next 
horizon.  But  they  obviously  lose  value  at  the  horizon 
boundary,  since  immediately  before  the  boundary 
they  are  all  the  resources  the  system  has,  while  after 
it  the  system  has  been  supplied  with  a  specified  addi¬ 
tional  number  of  planes.  Their  value  in  the  new  hori¬ 
zon  is  simply  the  expected  value  of  the  additional 
missions  they  make  possible  in  the  next  horizon, 
taking  into  account  the  fact  that  the  end  of  the  next 
horizon  might  conceivably  also  find  planes  sitting 
on  the  ground  unused.  The  expected  number  of  planes 
to  be  resupplied  at  the  beginning  of  each  horizon 
must  be  calculated,  and  this  depends  on  the  dispatch¬ 
ing  decisions  made  in  previous  horizons.  The  compu¬ 
tations  begin  with  a  list  of  numbers  which  in  essence 
predicts  the  number  of  sorties  to  be  flown  during 
each  horizon.  This  prediction  reflects  actual  past 
dispatching  decisions  and  forecasts  future  ones.  This 
prediction  is  updated  periodically  by  a  linear  pro¬ 
gramming  procedure  to  reflect  changes  based  on 
actual  experience  with  requests  and  actual  previous 
dispatching  decisions.  The  updating  will  probably 
be  done  once  per  horizon,  depending  on  how  much 
conditions  turn  out  to  differ  from  those  planned  for, 
how  much  computer  time  is  available  for  a  rather  de¬ 
manding  calculation,  and  so  on. 

The  procedure  taken  as  a  whole  is  suboptimal  in 
a  number  of  ways,  and  depends  on  a  number  of  special 
assumptions.  However,  it  should  be  a  good  first  ap¬ 
proximation.  As  it  becomes  desirable  to  improve  on 
it,  it  will  be  possible  to  change  parts  of  the  procedure 
piecemeal,  or  to  add  other  features  not  now  included. 

The  dispatching  rule 

The  dispatching  portion  of  JUDGE  is  developed 
under  a  set  of  assumptions  concerning  the  resources, 
admissible  decisions,  request  process,  target  values, 
and  mission  effectiveness  measures.  These  assump¬ 
tions  are  described  in  the  following  paragraphs. 

Resources.  The  aircraft  to  be  used  over  a  horizon 
are  all  of  the  same  type  and  loading  and  are  completely 
interchangeable.  Situations  involving  collections  of 
nonhomogeneous  aircraft,  in  which  the  substitutability 
of  one  type  for  another  depends  on  the  target,  would 
require  the  state  variable  representing  inventory  of 
remaining  sorties  to  be  multi-dimensional. 

Admissible  Decisions.  The  requests  JUDGE 
handles  are  of  the  “as  soon  as  possible”  variety,  and 
it  is  assumed  that  a  dispatching  decision  is  made  im¬ 
mediately  upon  receipt  of  each  request.  The  only  ad¬ 
missible  decisions  consist  of  assigning  to  the  mission 
a  number  of  sorties,  between  zero  and  the  number  of 


remaining  sorties.  Collecting  requests  or  delaying 
action  on  marginal  requests  is  not  permitted. 

The  Request  Process.  The  arrival  of  requests  over 
time  is  assumed  governed  by  a  Poisson  process  with 
a  known  rate  of  X  requests  per  hour.  The  substance 
of  this  assumption  is  that  over  a  very  small  time  in¬ 
terval  of  length  h,  the  probability  is  (1  —  Xh)  +  o(h) 
that  there  will  be  no  requests,  the  probability  is 
Xh  +  o(h)  that  there  will  be  one  request,  while  the 
probability  of  two  or  more  requests  is  negligible. 
(The  symbol  o(h)  means  a  quantity  very  small  relative 
to  h.)  These  probabilities  are  independent  of  any 
events  that  may  occur  outside  of  the  h-interval  under 
consideration.  The  request  rate  will  be  taken  to  be  a 
constant  over  any  particular  horizon,  although  allow¬ 
ing  X  to  vary  over  the  horizon  could  easily  be  incor¬ 
porated  into  the  calculation  if  dependence  of  X  on 
time  could  be  estimated. 

In  the  absence  of  evidence,  this  Poisson  hypothesis 
provides  a  plausible  assumption  regarding  the  arrival 
of  requests.  Certain  mathematical  results  indicate  that 
the  Poisson  process  is  an  appropriate  model  if  there 
are  many  potential  requestors  acting  independently. 
(See,  for  example,  Cox  and  Smith,  1 954.) 

Mission  Value  Functions .  Associated  with  each  re¬ 
quest  is  a  mission  value  function  that  specifies  an  im¬ 
mediate  reward  for  each  act  that  could  be  taken  in 
response  to  the  request.  Let  v  be  the  judged  value  of 
destroying  the  target,  and  let  r\{\)  be  the  extent  to 
which  a  mission  of  x  sorties  can  be  expected  to 
achieve  the  desired  effect.  The  mission  value  function, 
u(x),  is  taken  to  be  the  product: 

u(x)  =  vtj(x)  for  (x  =  0,  1,2,...). 

For  a  target  occupying  a  small  area  that  could  be 
destroyed  by  a  single  attacking  aircraft,  it  is  reasonable 
to  propose  that 

7](\)  =  I  —  (1  —  P)\ 

where  p  is  the  probability  that  one  aircraft  will  be 
successful.  Then  the  formula  above  represents  the 
probability  that  not  all  x  airplanes  miss  the  target. 
The  parameter,  p,  depends  on  many  factors  relating 
to  the  target,  aircraft,  weapons,  and  tactics,  but  cur¬ 
rent  technology  is  sufficiently  well  developed  to  pro¬ 
vide  reasonable  values.  In  an  implementation  of 
JUDGE,  it  is  possible  that  p  might  also  be  obtained 
through  the  judgment  of  qualified  personnel. 

As  a  result  of  this  form  for  tj(x),  the  mission 
value  functions  have  the  desirable  properties  that  the 
marginal  utility  of  sending  an  additional  plane  de- 
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ereases  for  increasing  x,  and  that  the  reward  for  not 
responding  to  a  request  is  zero. 

With  targets  of  significant  area,  say  troops  dis¬ 
persed  over  a  region  the  size  of  a  football  field,  the 
effectiveness  function  should  reflect  the  degree  to 
which  the  target  is  covered  by  lethal  ordnance.  By 
making  some  coarse  assumptions,  the  same  simple 
form  for  r?(x)  ean  be  rationalized.  Let  T  denote  the 
target  area,  let  L  represent  the  lethal  area  of  the 
weapons  a  single  aircraft  deposits  on  the  target,  and 
let  p,  be  the  probability  that  a  plane  successfully 
hits  the  target.  If  we  subdivide  the  target  into  areas 
of  size  L  and  assume  that  a  plane  hitting  the  target 
hits  any  particular  subdivision  with  probability  L/T, 
then  the  probability  that  any  subdivision  is  killed  is 
(P,L)/T.  The  probability  that  any  particular  sub¬ 
division  is  killed  by  an  attack  of  x  aircraft  is  then 
given  by 

i?(x)  =  1  -  (1  -  p)\ 

where 


and  this  is  also  the  total  expected  portion  of  the 
target  destroyed. 

Sensitivity  to  the  number  of  sorties  ean  be  varied 
by  introducing  a  second  parameter  and  writing  the 
mission  effectiveness  formulas  as 


correlated,  we  shall  indicate  a  marginal  distribution 
for  target  values  and  a  conditional  distribution  for 
effectiveness  parameters.  Denoting  the  two  random 
variables  by  V  and  P,  respectively,  define: 

F(v)  =  Prob(V  <  v), 

and 

G(p|v)  =  Prob(P  ^  p|V  =  v). 

These  distributions  are  assumed  known;  the  fore¬ 
casting  problem  will  be  discussed  in  the  following 
subsection. 

Derivation  of  the  Dispatching  Rule.  Within  a 
horizon,  the  state  of  the  system  may  be  described  by 
specifying  the  number,  n,  of  remaining  sorties  avail¬ 
able  for  dispatching  and  the  amount  of  time,  t,  left 
until  the  horizon  is  over.  The  cost  associated  with 
dispatching  sorties  is  achieved  by  attributing  a  value, 
Wn(t),  to  being  in  the  state  (n,t).  This  value  is  a  mea¬ 
sure  of  the  total  expected  utility  associated  with  the 
dispatching  decisions  to  be  made  over  the  remaining 
t  hours  of  the  horizon  when  there  are  n  sorties  that 
ean  be  used.  Then  the  priee  of  x  sorties  in  the  state 
(n,t)  is  Wn(t)  -  Wn^x(t). 

tf  a  request  having  attributes  v  and  p  is  received 
when  the  state  is  (n,t),  the  best  decision  is  to  choose 
the  value  of  x,  (x  =  0,  1 , . . .  ,n),  that  yields  the  maxi¬ 
mum  difference  between  reward  and  cost,  the  value 
of  x  that  maximizes 

u(x:v,p)  +  Wn  x(t)  -  Wn(t). 


7](\)  =  1  —  (  1  —  P)X 


Since  the  decision  does  not  affeet  the  last  term  in 
the  above  expression,  it  may  be  dropped.  The  quantity 


We  have  explored  hypothetical  effectiveness  func¬ 
tions  generated  this  way  with  a  in  the  range  of  0.8 
to  1.4.  Although  we  offer  no  interpretation  for  the 
extra  parameter,  a  function  of  this  form  should  be 
capable  of  fitting  a  wide  variety  of  effectiveness  func¬ 
tions  that  might  be  derived  from  very  detailed  analy¬ 
ses.  For  simplicity  we  shall  assume  that  the  mission 
effectiveness  functions  are  specified  by  the  choice 
of  a  single  parameter,  p. 

In  order  to  keep  in  mind  that  the  mission  value 
function  depends  on  both  the  estimated  target  value 
and  the  parameter  of  the  effectiveness  function,  a 
mission  value  function  will  be  indicated  by  u(x:v,p). 

In  developing  the  dispatching  rule,  the  target  values 
and  effectiveness  parameters  will  be  treated  as  random 
variables.  Sinee  these  variables  may  very  well  be 


max  {u(x:v,p)  +  Wn_x(t)} 
O^x^n 


represents  the  sum  of  the  immediate  reward  and  the 
value  of  the  state  resulting  from  the  best  decision  in 
the  event  that  a  request  with  characteristics  v  and  p 
is  received  when  the  system  is  in  state  (n,t). 

Under  the  notion  that  the  target  value  and  ef¬ 
fectiveness  functions  associated  with  a  request  are 
selected  randomly,  the  expected  value  of  the  reward, 
plus  the  value  of  the  state  resulting  from  the  action 
taken  when  a  request  arrives  at  state  (n,t),  is 


max  {u(x:v,p)  +  Wn_x(t)}  dF(v)  dG(p|v). 
O^x^n 


v  p 


L 
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Taking  h  to  be  a  very  small  number,  suppose  that 
there  are  n  sorties  available  in  a  horizon  that  ends  in 
t+h  hours.  Given  that  a  request  arrives  in  the  next  h 
hours,  the  expression  above  represents  the  value  of 
state  (n,t  +  h).  On  the  other  hand,  if  there  are  no 
requests  in  the  small  time  interval,  the  sum  of  the  ex¬ 
pected  rewards  to  be  earned  over  the  t  +  h  hours  is 
the  same  as  that  for  the  slightly  shorter  horizon  of 
length  t  hours,  and  then  Wn(t  +  h)  =  Wn(t).  By  as¬ 
sumption,  the  probabilities  of  a  request  and  no  re¬ 
quest  in  the  small  interval  are,  respectively,  Xh  and 
( 1  —  Xh).  Weighting  the  two  expressions  for  Wn(t  +  h) 
by  the  corresponding  probabilities  gives 

Wn(t+  h)  =  (1  -  Xh)  Wn(t)  +  Xh  |  [  max  {u(x:v,p) 

J  J  O^x^n 
v  p 

+  Wn-x(t)}dF(v)dG(p|v). 


Following  the  standard  procedure  of  subtracting 
Wn(t)  from  both  sides  of  the  equation,  dividing  by 
h,  and  then  taking  limits  as  h  beeomes  small,  we  ob¬ 
tain  the  differential  equation: 

-W~+\Wn(t)  =  X  [  [  max  {u(x:v,p) 
dt  J  J  O^x^n 

v  p 

+  Wn_x(t)}  dF(v)  dG(p|v), 


which  will  be  referred  to  as  the  “value  equation/’ 
For  any  horizon,  a  series  of  these  equations  are  solved 
recursively  starting  with  n  =  1  and  going  up  to  the 
number  of  sorties  allocated  for  use  in  the  horizon. 
The  recursive  solution  is  neeessary  beeause  for  any 
value  of  n,  the  solution  depends  on  the  Wm(t)  functions 
for  m  <  n. 

The  particular  solutions  depend  on  a  set  of  bound¬ 
ary  conditions  as  follows:  the  notion  that  the  utility 
of  no  sorties  available  at  any  time  is  zero  is  em¬ 
bodied  in  the  boundary  condition 

W0(t)  =  0  for  (t  2*  0). 

In  addition,  a  boundary  condition  is  needed  for  each 
value  of  n  considered  to  specify  the  value  of  having 
n  sorties  left  over  when  the  horizon  ends.  For  the  last 
horizon,  leftover  sorties  are  useless  so  that  in  this 


speeial  ease  it  would  be  reasonable  to  impose  the 
conditions: 

Wn(0)  =  0  for  (n  =  1,2,...). 

For  other  horizons,  boundary  values  should  be  re¬ 
lated  to  the  marginal  value  of  having  additional 
sorties  available  in  the  following  horizon.  The  use  of 
these  boundary  values  to  eonneet  one  horizon  with 
the  following  one  will  be  diseussed  under  planning. 

The  dispatehing  rule  itself  is  a  byproduct  ob¬ 
tained  in  solving  the  value  equations,  and  consists 
of  a  table  of  the  optimizing  x’s  as  functions  of  the 
parameters  v,  p,  n,  and  t.  However,  in  an  imple¬ 
mentation  of  JUDGE,  it  would  be  more  practical  to 
retain  just  the  solutions,  Wn(t),  as  a  two-dimensional 
table  with  some  suitable  grid  for  the  time  parameter. 
Then  the  optimizing  decisions  for  any  v  and  p  com¬ 
bination  eould  be  readily  computed  as  required. 

A  fruitful  way  of  characterizing  the  dispatehing 
rule  is  to  define  Rx(n,t)  as  the  set  of  points  (v,p)  such 
that  the  optimal  ehoiee  is  to  dispateh  x  sorties  for  a 
request  with  attributes  v  and  p  received  when  the 
system  state  is  (n,t). 

A  boundary  point  between  Rx_,(n,t)  and  Rx(n,t) 
may  be  determined  by  fixing  p  and  solving  for  the 
value  of  v  that  yields  the  same  total  of  immediate 
expected  reward  plus  expected  value  of  the  result¬ 
ing  state  when  either  x— 1  or  x  sorties  are  dispatched. 
That  is,  for  a  particular  mission  effectiveness  function 
r}(\)  (which  is  determined  by  the  ehoiee  of  p)  solve 
for  the  value  of  v  that  satisfies 

vij(x-I)  +  Wn_x+1(t)  =  vtj(x)  +  Wn_x(t). 

The  solution,  denoted  by  vx  is 


..  _  Wn_x+t(t)  -  Wn_x(t) 

T?(X)  "  T?(X— I) 

For  v  <  vx,  it  is  better  to  send  x— I  sorties;  for  v  >  vx, 
sending  x  sorties  is  the  preferred  action.  That  is, 
vx  is  the  minimum  value  of  v  for  which  x  sorties 
would  be  dispatehed  for  a  request  whose  mission 
success  parameter  is  p  and  the  state  is  (n,t). 

Some  eare  must  be  taken  beeause  the  sequence  of 
vx’s  obtained  in  this  way  may  not  be  increasing.  It 
eould  happen  that  vx+,  <  vx.  Such  a  reversal  would 
indicate  that  there  is  no  value  of  v  sueh  that  dispatch¬ 
ing  x  sorties  is  the  optimal  aetion.  Then  the  ehoiee 
is  between  x— I  and  x-FI,  and  a  new  value  of  vx+, 
must  be  determined  as 


! 


Wn_x+,(t)  —  Wn_„_,(t) 

x+l  7j(X+l)-7j(X-l) 


Now  it  must  be  checked  that  vx+1  >  vx_,.  If  not, 
x— I  sorties  would  never  be  dispatched,  and  vx+1 
must  be  recomputed  by  comparing  the  results  of  dis¬ 
patching  x— 2  and  x+1  sorties,  provided  that  x  5*  2. 
This  checking  and  recomputing  process  continues 
until  either  there  are  no  more  reversals  or  we  have 
found  the  point  on  the  boundary  of  R0(n,t)  and 
R\+,(n,t). 


Porometef  of  •ff*ctiven«s  function  (p) 

Figure  l  —  Typical  dispatching  decisions  for  fixed  state  (n,t) 
for  a  combination  of  (v,p)  *RX,  dispatch  x  aircraft 

To  illustrate  how  the  sets  Rx(n,t)  might  look  on 
the  (v,p)  plane,  Figure  I  was  constructed  for  the  mis¬ 
sion  value  function  u(x:v,p)  =  v(  I  —  ( 1  — p)x).  The 
values  of  Wn(t)  were  taken  from  a  numerical  example 
whose  solutions  to  the  value  equations  are  shown  in 
Figure  2,  To  generate  the  curves  of  Figure  1,  n  was 
chosen  to  be  20,  while  t  was  set  at  2,0  at  the  end  of 
horizon  I.  It  was  assumed  that  only  even  numbers 
of  sorties  would  be  sent  on  any  mission,  so  that  for 
any  p,  vx  was  calculated  from 


v 


X 


^22~x(0  W20_x(t) 

(|-P)*~2“  (  1— P)X 


for  x  =  2,  4, .  . 


,10. 


These  curves  reflect  the  notion  that  the  higher  the 
value  of  the  request,  for  a  fixed  parameter,  p,  the 
more  sorties  we  are  willing  to  expend  in  an  effort  to 
achieve  success.  The  general  U-shapc  of  the  regions  in 
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Figure  I  arc  attributable  to  the  behavior  of  7/(x)  as 
a  function  of  p.  For  missions  of  constant  v,  the  ad¬ 
ditional  expected  reward  for  dispatching  an  additional 
aircraft  is  proportional  to  7](x+\ )  —  7?(x).  As  a  function 
of  p,  this  difference  increases  for  small  values  of  p, 
reaches  a  maximum,  depending  on  x,  and  then  de¬ 
creases.  Suppose  that  the  optimum  action  were  to 
send  x  sorties  on  a  mission  with  value  v  and  some 
small  success  parameter,  p.  If  p  were  a  bit  larger, 
the  additional  reward  for  sending  another  aircraft 
might  be  great  enough  to  justify  another,  according 
to  our  expected  value  criterion.  But  if  p  is  large,  the 
improvement  in  reward  to  be  gained  by  another  air¬ 
craft  is  insensitive  to  changes  in  p.  In  the  limiting 
case  where  p  =  1 ,  there  would  never  be  any  reason 
for  dispatching  more  than  one  aircraft. 

If  n  were  smaller  or  t  were  larger,  the  region  Rx(n,t) 
would  be  moved  upward,  because  under  these  the 
price  of  sorties  would  be  higher. 

The  value  equation  may  be  written  in  terms  of  the 
regions,  Rx(n,t),  as 


dWn(t) 

dt 


+  k 


I  - 


J  J  dF(v)  dG(p| v) 

(v,p)  e 
R«(n,j) 


Wn(t) 
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[u(x:v,p)  +  Wn  x(t)]  dF(v)  dG(p|v). 


(v,p)  e 

Rx(n,t) 


It  is  understood  that  a  and  (3  are  specific  for  particular 
n  and  j.  They  may  be  evaluated  numerically  by  plac¬ 
ing  grids  over  the  domains  of  v  and  p.  According  to 
the  approximation  scheme,  they  too  are  constant  over 
the  A  interval,  and  the  approximate  form  for  the  value 
equation  is 


Approximate  Solutions  to  the  Value  Equation. 
Even  with  the  simplest  assumptions  about  the  forms 
of  u,  F,  and  G,  it  would  be  difficult  to  obtain  exact 
solutions  to  the  value  equation  because  the  coef¬ 
ficient  of  Wn(t)  and  the  right-hand  side  are  very 
complicated  functions  of  t.  A  simple  and  satisfactory 
solution  to  this  problem  is  available  by  approximat¬ 
ing  the  true  solutions  with  step  functions. 

To  compute  approximate  solutions,  the  time  horizon 
is  divided  into  increments  of  length  A,  and  approxi¬ 
mate  values  of  Wn(t)  are  determined  at  the  points 
A,  2A,  3A, .  . .  ,  under  the  assumption  that  Wn(t)  is 
constant  over  the  half-open  interval  fj A,  (j-f-l)A). 
Let  the  approximating  value  for  Wn(t)  for  t  c[jA, 
(j-Fl)A)  be  denoted  by  Wn(jA). 

In  carrying  out  the  computations,  one  would  start 
with  j  =  1  and  calculate  Wm(A)  for  each  m  from  1  up 
to  the  maximum  value  desired.  Then  j  would  be 
£tepped  and  the  process  repeated,  calculating  the 
Wm(2A)  values,  etc. 

Consider  the  computation  of  Wn[(jTl)A].  The 
quantities,  Wm(jA),  (m  =  0,  1 , . .  .  ,n)  have  already 
been  obtained  and  are  stored.  Since  the  Wm(t)’s  are 
taken  to  be  constant  over  the  A  interval,  the  decision 
rule  is  constant  over  the  interval  also.  The  sets, 
Rx(nj),  of  points,  (v,p),  such  that  x  sorties  would  be 
dispatched  may  be  determined.  For  notational  con¬ 
venience  define: 


dWn(t) 

dt 


+  «Wn(t)  = 


P- 


The  solution  to  this  equation  is 


Wn(t)  =  ce  at  + 


a.  ■  £ 


where  c  is  an  arbitrary  constant.  The  constant  is 
evaluated  from  the  boundary  condition  supplied  by 
the  known  value  of  Wn(jA).  Thus, 


e*j\ 


and  Wn[(jTl)A]  is  given  by  the  recursive  relationship: 


W„[(j+1)A] 


e-aA 


a  =  x[l-  J  J  dF(v)  dG(p|v)J, 

Jv,p)e 

Ro(n,j) 


0  =  A 


Rx(n,j) 


[u(x:v,p)  +  Wn  -x(jA)] 


dF(v)  dG(p|v). 


The  estimates  of  Wn(t)  obtained  by  this  method  will 
be  smaller  than  the  true  values.  Upper  bounds  can 
also  be  obtained  by  using  Wm’s  at  (j  -F  1)A  rather  than 
jA  in  the  computation  of  a  and  (3.  In  developing  the 
Rx(nj)  sets,  a  preliminary  estimate  of  Wn[(j  -F  1)A] 
is  required  (this  is  what  we  are  trying  to  compute), 
but  this  can  be  done  by  extrapolating  from  W  values 
associated  with  surrounding  combinations  of  j  and  m. 
It  is  not  difficult  to  develop  compromise  schemes 
that  give  estimates  between  the  upper  and  lower 
bounds.  All  such  variations,  including  the  upper  and 
lower  bounds,  are  asymptotically  correct  (for  large 
t)  so  that  there  is  good  control  over  error.  Accuracy 
is  improved  by  making  A  small,  but  the  amount  of 
computation  varies  directly  with  the  number  of  points 
on  the  time  grid.  Computational  experience  has  shown 
that  some  refinement_in  the  method  of  choosing  re¬ 
placements  for  the  Wm(jAfs  for  use  in  computing 
a  and  (3  can  make  up  for  a  rather  coarse  grid. 
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Figure  2  displays  approximate  solutions  to  the  value 
equation  for  two  horizons  covering  the  last  four  hours 
of  a  day.  The  time  scale  is  in  hours  remaining,  and  the 
boundary  between  the  two  horizons  is  at  2.0  hours. 
In  both  horizons,  the  number  of  available  aircraft 
was  set  at  20  (curves  for  odd  numbers  of  remaining 
sorties  have  been  omitted).  Since  time  0  represents 
the  end  of  the  flying  day,  the  boundary  conditions 
were  set  to  reflect  zero  value  for  any  unused  sorties 
at  time  0.  The  boundary  values  for  the  next  earlier 
horizon  (horizon  2)  were  set  in  accordance  with  the 
conclusions  reached  in  the  next  subsection.  Ad¬ 
ditional  details  about  the  specific  example  used  to 
generate  the  data  in  Figure  2  are  given  at  the  end  of 
Sec.  III. 

Models  similar  to  the  one  presented  in  this  section 
have  been  published  by  Kincaid  and  Darling  (1963) 
and  Kaufman  (1963). 

The  planning  technique 

T  he  purpose  of  the  planning  portion  of  JUDGE  is 
to  set  values  for  the  number  of  sorties  available  at 
the  beginning  of  each  horizon.  The  planning  computa¬ 
tion  will  probably  be  done  once  per  horizon.  For 
specificity,  the  following  paragraphs  describe  proce¬ 
dures  appropriate  for  the  first  such  calculation  of  the 
day;  minor  modifications  are  required  for  subsequent 
updatings. 

In  our  notational  scheme,  the  horizons  will  be 
numbered  from  the  last  occurring  (horizon  number  1) 
to  the  earliest.  Such  “backward  numbering”  is  slight¬ 
ly  more  convenient  than  numbering  horizons  in  their 
natural  sequence  and  is  analogous  to  our  use  of  “t” 
in  the  previous  discussion  to  represent  the  time  re¬ 
maining  in  a  horizon.  If  there  are  r  horizons  in  the 
flying  day,  the  result  of  the  planning  stage  is  an  r- 
dimensional  vector  y  =  (yH  y2, . . . ,  yr),  where  the  ith 
component  represents  the  number  of  sorties  planned 
to  be  available  when  the  ith  horizon  begins.  These 
numbers  will  not  be  exactly  realized  because  the  re¬ 
sults  of  the  dispatching  procedure  and  the  aircraft 
recovery  are  subject  to  uncertainty,  and  because  the 
planning  technique  is  based  on  certain  approxima¬ 
tions. 

Since  we  must  now  consider  a  number  of  horizons, 
the  Wn(t)  functions  will  be  superscripted  with  the 
appropriate  horizon  indices.  Thus  W„(t)  will  be  the 
expected  value  of  the  state  (n,t)  in  the  ith  horizon.  Also 
let  the  length  of  the  ith  horizon  be  denoted  by  t,. 

In  developing  the  plan,  we  assume  that  there  is  a 
fixed  number,  s,  of  aircraft  that  are  to  be  used  for 
close  air  support  during  the  day.  The  object  is  to  de¬ 
termine  y,  subject  to  restrictions  about  the  number 
of  sorties  that  can  be  flown,  that  maximizes  the 
quantity: 


i  w,  a,). 

Pi 

This  objective  function  is  appropriate  since  one  term 
in  the  sum  represents  the  total  expected  value  of  the 
sorties  available  within  the  associated  horizon.  Then 
the  sum  over  horizons  is  a  measure  of  the  expected 
value  to  be  gained  from  the  allocation  y. 

It  is  clear  that  the  planning  technique  should  de¬ 
pend  on  the  dispatching  procedure,  because  it  is  im¬ 
possible  to  plan  without  taking  into  account  the  ef¬ 
fects  of  the  dispatching  method.  This  dependence  is 
evident  in  our  choice  of  objective  function.  On  the 
other  hand,  the  dispatching  rule  depends  on  the  plan¬ 
ning  technique  in  the  computation  of  the  W‘(t)  func¬ 
tions.  As  discussed  previously,  the  dispatching  rule 
for  the  ith  horizon  depends  on  the  boundary  condi¬ 
tions  W„(0),  for  (n  =  1 , 2, . . .  ,y,). 

The  issue  is:  what  should  these  boundary  conditions 
be?  Taking  them  to  be  zero  is  unsatisfactory  because 
this  would  place  an  unrealistically  low  price  on  mis¬ 
sions  at  the  end  of  the  horizon.  Another  alternative 
is  to  make  the  value  of  n  sorties  at  the  end  of  a  horizon 
the  same  as  the  value  of  n  sorties  at  the  beginning 
of  the  next  horizon.  That  is,  to  set  W^'(O)  =  W*,^). 
But  this  would  place  too  high  a  price  on  missions  at 
the  end  of  the  (i+l)st  horizon  in  view  of  the  imminent 
resupply. 

Sincp  unused  sorties  can  be  carried  over  to  the  next 
horizon,  the  boundary  values  should  reflect  their 
expected  utility  in  the  next  horizon.  Suppose  that 
the  planned  number  of  available  sorties  for  the  ith 
horizon  is  yh  Then  the  boundary  conditions  for  the 
next  earlier  horizon  should  be 

wr(O)  =  wj  +n(t,)  -  w‘  (t.) . 

1  i 

The  right-hand  side  of  this  equation  is  a  measure  of 
the  additional  utility  of  n  sorties  brought  from  the 
(i-h I )st  horizon  to  the  next. 

Suppose  that  there  are  n  sorties  still  left  h  hours 
before  the  end  of  the.  (i+*l)st  horizon,  where  h  is  a 
very  small  number.  The  cost  of  dispatching  x  sorties 
under  these  conditions  would  be 

wr(h)  -  wr_*x<h)  =  I  Wj  +n(t,)  -  W‘  (t.) 

L  I  1  J 

I  Wy  +n_X(t,)  -  Wy  (t|)l  =  Wy!  +  „(t()  -  . 

L  i  i  J  j  i 
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Thus,  this  way  of  setting  the  boundary  conditions 
satisfies  our  intuitive  feeling  that  the  cost  of  x  sorties 
should  be  continuous  across  horizon  boundaries. 

The  interdependence  between  planning  and  dis¬ 
patching  suggests  that  the  plan  be  arrived  at  by  an 
iterative  procedure  such  as  the  following.  The  compu¬ 
tation  is  begun  with  an  initial  guess  at  the  allocation 
vector  _y.  Based  on  this  starting  point,  the  values 
W^tj),  associated  with  various  numbers  (say  from 
1  to  s)  of  available  sorties  at  the  start  of  each  horizon 
are  calculated  by  the  dispatching  rule  computation 
for  use  in  the  objective  function  of  the  planning 
algorithm  (still  to  be  described).  The  planning  algo¬ 
rithm  gives  a  new  y  vector  that  satisfies  the  sortie 
availability  constraints  and  maximizes  the  objective 
function. 

However,  the  boundary  conditions  used  to  calcu¬ 
late  the  Wm(tj)  values  depend  upon  the  original  _y 
vector.  If  the  new  allocation  is  very  different  from 
the  original,  the  W^q)  values  corresponding  to  the 
new  allocation  might  differ  considerably  from  the 
original  set  of  values,  so  that  the  new  allocation 
would  not  really  be  optimal.  The  solution  is  to  cal¬ 
culate  the  new  set  of  end-of-horizon  values  based  on 
the  new  allocation  and  use  them  in  the  planning  algo¬ 
rithm  to  obtain  a  third  allocation.  This  iterative  proce¬ 
dure  could  be  carried  out  until  two  successive  alloca¬ 
tions  are  fairly  close. 

Although  there  has  not  yet  been  any  computational 
experience,  the  convergence  should  be  rapid  and  it 
is  doubtful  that  the  process  of  recalculation  would 
be  (or  should  be)  carried  out  more  than  once.  This 
conjecture  is  supported  by  numerical  results  indicat¬ 
ing  that  end-of-horizon  values  are  not  very  sensitive 
to  the  boundary  conditions  from  which  they  were 
obtained.  (Compare  the  sets  of  end-of-horizon  values 
for  the  two  horizons  in  Figure  2.  The  only  difference 
between  the  horizons  is  in  the  boundary,  conditions.) 

One  difficulty  arises  in  deriving  the  boundary  con¬ 
ditions  for  a  horizon  from  the  end-of-horizon  values 
of  the  next  later  horizon.  To  allow  a  full  range  of 
possibilities  for  the  x  vector,  the  planning  algorithm 
should  be  supplied  with  values  W^tj)  running  from  1 
to  s  for  each  of  the  r  horizons.  It  is  natural  to  take 
all  s  boundary  values  for  the  last  horizon  to  be  zero. 
The  boundary  values  for  the  next  to  last  horizon 
(horizon  2)  depend  on  the  current  y,  since  we  take 
W*(0)  =  W^U;)  -  Wy^t).  This  provides  only  s-y, 
boundary  values  for  horizon  2.  Continuing  in  this 
way,  we  would  be  able  to  calculate  only  s— y,— y2 
boundary  values  for  horizon  3,  and  so  on.  A  satis¬ 
factory  way  to  avoid  running  out  of  boundary  condi¬ 
tions  is  to  set  the  missing  boundary  values  for  a 


horizon  equal  to  the  highest  legitimate  value  avail¬ 
able  from  the  previous  horizon.  For  example,  in 
horizon  2  take 

W*(0)  -  Ws%  (O)  =  WJ(tl)  -  W  *  (t/) 

i  s  i 

for  (m  =  s— yx+\ , . . .  ,s). 

A  Planning  Algorithm.  The  vector,  x,  which  is 
chosen  to  maximize 

t  W‘  (4)  , 

i  =  l  * 

is  subject  to  constraints  that  reflect  the  limited  num¬ 
ber,  s,  of  aircraft  and  the  distribution  of  takeoff-to- 
ready  times.  The  planning  algorithm  could  be  formu¬ 
lated  as  a  stochastic  programming  problem,  but  such 
a  formulation  would  be  hopeless  from  the  computa¬ 
tional  standpoint.  Therefore,  we  shall  retreat  by 
writing  the  constraints  in  terms  of  expected  values. 

Since  we  are  operating  as  though  resupply  occurs 
only  at  discrete  points  in  time;  a  discrete  representa¬ 
tion  of  the  takeoff-to-ready  time  distribution  will  be 
used.  The  most  useful  form  in  which  to  express  the 
distribution  is  achieved  by  defining: 

qy  =  Frob  (an  a/c  launched  in  horizon  j  will  not  be¬ 
come  ready  by  horizon  i) 
for  (i  <  j,  j  =  2,. . .  ,r), 

qji  =1  for  (j  =  I , . . .  ,r). 

The  double  subscript  notation  allows  us  to  specify 
a  different  distribution  for  each  launching  horizon  0) 
if  necessary.  This  will  be  necessary  if  the  horizons 
are  not  all  of  the  same  length.  For  each  j,  there  is  a 
distribution  of  takeoff-to-ready  times.  Then  qu  is 
the  sum  of  the  tail  of  the  distribution  over  horizons 
occurring  later  than  i  (horizons  with  indices  less  than 
i).  Setting  qjj  =  1  is  equivalent  to  assuming  that  an 
aircraft  cannot  be  used  twice  within  one  horizon. 

For  planning  purposes,  assume  that  all  sorties  made 
available  within  a  horizon  will  actually  be  used  during 
that  horizon.  Since  JUDGE  is  intended  for  use  under 
conditions  of  resources  severely  limited  relative  to 
expected  demands,  this  assumption  should  not  be 
far  from  the  truth. 

The  total  expected  number  of  sorties  that  are  un¬ 
available  (in  the  air  or  being  recovered)  by  the  end  of 
horizon  i  is  the  sum  over  each  horizon  from  r  down  to 
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i  of  the  product  of  sorties  launched  during  the  horizon 
times  the  proportion  of  those  sorties  that  have  not 
yet  returned  to  the  ready  state.  This  number  of 
sorties  must  be  less  than  the  total  number  of  aircraft, 
s.  Then  the  set  of  r  constraints  (one  for  each  horizon) 
have  the  form: 

V  qsjyj  «  S  for  (i  =  I . r)  . 


Although  only  integer  solutions  to  this  programming 
problem  are  desired,  in  view  of  the  other  approxima¬ 
tions  being  made,  one  should  not  object  to  rounding 
fractions  if  s  is  not  a  small  number. 

An  inconvenience  arises  because  the  objective 
function  is  non-linear  in  the  components  of  x,  but  this 
can  be  remedied  by  reformulating  the  problem  in 
terms  of  another  set  of  variables.  Suppose  that  x  is  an 
allocation  vector.  For  each  j,  (j  =  define 


'I  'f  k  y, 

Zjk  =*<  yj  -  k  —  I  if  y,  <  k  <  y,  +  I 
„0  if  y,  +  I  k 


Going  the  other  way,  given  zJk  0  =  I , . . .  ,r;  k  = 
1 , . . .  ,s),  then 

yj  =  2  zik  for  0  =  I . r)  . 

k  =  l 


Also  define 


C)k  =  Wj (t,)  -  wi_,(tj)  for  o  =  1 ....  .r  ; 

k  =  I . s)  . 

The  variable  zJk  has  a  value  of  I  if  the  k,h  sortie  should 
be  assigned  to  the  jlh  horizon  and  0  if  not.  The  coeffi¬ 
cient  cJk  is  the  incremental  value  of  assigning  the 
klh  sortie  in  the  jth  horizon. 

Consider  the  linear  programming  problem: 

r  s 

maximize  Z  =  Y  V  cjkzjk  , 

k=t 


subject  to  V  qu  £  zjk  ^  s  (i  —  I . r) 

f=i  k  =  1 


0  *£  zik  ^  l  Ci  =  I  *  •  •  •  t; 

k=  1 . s). 


In  any  reasonable  situation,  one  would  expect  that 
Cjk  ^  Cj(k_,)  (decreasing  marginal  utility  of  additional 
sorties),  which  would  imply  that  zJ(k_,>=  I  if  zJk  >  0. 
This  assures  us  that  a  solution  to  the  above  problem 
would  not,  for  example,  refuse  to  assign  the  twelfth 
sortie  and  simultaneously  indicate  that  there  should 
be  a  thirteenth  one,  and  that  there  would  be  at  most 
one  fractional  zJk  for  any  j.  Then  the  two  programming 
problems  are  equivalent.  Very  efficient  computer 
codes  for  solving  quite  large  linear  programming 
problems  exist,  and  good  methods  for  solving  the 
“zero-one”  problem  are  becoming  available. 

Forecasts  of  future  mission  requests  are  incor¬ 
porated  into  JUDGE  through  the  specification  of 
request  rates  and  the  joint  distributions  of  the  mission 
characteristics  v  and  p  for  each  horizon.  These  are 
used  directly  in  computing  the  dispatching  rule  and 
indirectly  in  planning,  since  the  planning  technique 
uses  the  dispatching  calculation. 

In  an  implementation  of  JUDGE,  it  is  likely  that 
the  planning  computation  would  be  carried  out  several 
times  in  the  course  of  a  single  day,  perhaps  at  the 
beginning  of  each  horizon.  This  would  make  it  pos¬ 
sible  to  incorporate  forecast  modifications  if  they 
were  indicated  after  the  beginning  of  a  day.  It  would 
also  permit  the  system  to  adapt  to  the  recovery  of  air¬ 
craft  that  have  been  dispatched  in  earlier  horizons. 
When  redoing  the  planning  calculation,  the  current 
fleet  status  would  be  incorporated  into  the  con¬ 
straint  set  of  the  linear  program,  and  the  new  fore¬ 
casts  would  be  used  in  the  dispatching  computations 
that  provide  the  objective  function. 

Although  a  formal  forecasting  system  for  JUDGE 
has  not  yet  been  developed,  experience  in  previous 
days,  modified  by  knowledge  of  daily  plans,  should 
provide  reasonable  estimates  of  future  demands. 

A  Numerical  Example  of  the  Dispatching  Computa¬ 
tion .  A  computer  program  capable  of  calculating  the 
dispatching  rule  for  simple  examples  has  been  written 
in  order  to  examine  certain  aspects  of  the  dispatch¬ 
ing  rule.  This  program  was  used  to  generate  the  data 
for  Figure  2.  In  the  interest  of  simplicity,  the  program 
was  designed  to  operate  with  a  simpler  mission  value 
and  probability  structure  than  that  described  previous¬ 
ly.  Instead  of  working  with  an  analytical  formulation 
of  u(x:v,p)  and  a  joint  distribution  of  v  and  p,  the 
program  can  accommodate  up  to  eight  arbitrary  mis¬ 
sion  value  functions  in  the  form  of  a  table.  These 
functions  are  indexed  by  a  single  parameter.  The 
distribution  functions  F  and  G  are  replaced  by  a  set 
of  up  to  eight  probabilities  representing  a  distribution 
over  the  indices  of  the  mission  value  functions. 

For  the  numerical  results  to  be  discussed,  eight 
mission  value  functions  were  used.  None  gave  an 
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improvement  in  immediate  reward  for  more  than  eight 
sorties  or  offered  a  reward  greater  than  8.0  when  the 
maximum  of  eight  sorties  was  dispatched.  All  re¬ 
sults  are  based  on  boundary  conditions  of  Wn(0)  =  0; 
n  was  carried  up  to  48  aircraft,  and  t  was  taken  to 
5.0  hours.  The  data  used  to  plot  the  left  side  of  Figure 
2  represent  a  portion  of  the  results  for  the  particular 
case  of  X  =  6.0  requests  per  hour.  (The  time  units 
are  rather  arbitrary;  by  relabeling  the  time  scale 
this  would  be  equivalent  to  a  two-hour  horizon  with 
a  request  rate  of  15  per  hour,  or  any  other  combina¬ 
tion  for  which  the  expected  number  of  requests  in 
the  horizon  is  30.) 

The  primary  outputs  of  the  program  are  the  solu¬ 
tions  to  the  value  equations  and  the  dispatching  rule 
in  the  form  of  a  three-dimensional  table.  This  table 
specifies  the  number  of  aircraft  to  be  sent  as  a  func¬ 
tion  of:  the  number  of  remaining  sorties,  the  time  left 
in  the  horizon,  and  the  index  number  of  the  mission 
value  function  appropriate  to  the  request.  In  addition, 
the  program  has  the  ability  to  test  the  dispatching 
rule  with  sequences  of  requests  generated  by  the 
Monte  Carlo  method. 

Table  I  illustrates  how  the  dispatching  rule  behaves 
with  respect  to  a  particular  type  of  mission  request 
(i.e.,  a  fixed  mission  value  function)  as  the  request 
rate,  number  of  sorties  available,  and  time  remaining 
are  varied.  T  he  sequence  of  appropriate  decisions  as 
a  function  of  time  is  given  for  three  values  of  X  and 
six  values  of  n.  As  one  would  expect,  the  generosity 
with  aircraft  increases  with  the  number  of  sorties  left 
and  as  the  end  of  the  horizon  approaches,  but  de¬ 
creases  with  higher  request  rates. 


The  data  of  Table  2  are  the  result  of  a  Monte  Carlo 
experiment  designed  to  examine  the  sensitivity  of  the 
dispatching  rule  to  errors  in  forecasting  the  request 
rate.  Five  values  of  X  were  used.  The  column  labeled 
“Rule  Value”  contains  the  values  of  W4K(5.0)  ob¬ 
tained  from  the  value  equation  in  the  process  of  cal¬ 
culating  the  rules.  When  the  forecast  is  accurate, 
these  numbers  represent  the  expected  total  value  of 
all  requests  satisfied.  The  remaining  column  head¬ 
ings  indicate  the  actual  rates  that  governed  the  arrival 
of  mission  requests  in  the  Monte  Carlo  tests.  The 
entries  in  these  columns  are  the  averages  of  values 
actually  attained  for  missions  dispatched  for  the  indi¬ 
cated  combinations  of  forecasted  and  actual  rates. 
Within  each  column,  the  highest  score  was  obtained 
when  the  forecast  was  correct,  but  it  also  appears 
that  the  dispatching  rule  can  tolerate  fairly  high  fore¬ 
cast  errors  without  a  great  deal  of  degradation  in 
the  performance  measure. 

Table  2  — Values  attained  for  various  forecasted 
and  actual  request  rates 


\ 

Rule 

Value 

Actual  Request  Rate 

2.0 

4.0 

6.0 

8.0 

12.0 

2.0 

46.6 

45.7 

60.4 

62.4 

64.2 

66.0 

4.0 

61.7 

42.9 

61.2 

66.9 

71.1 

73.9 

6.0 

68.5 

39.0 

59.2 

68.7 

73.0 

78.9 

8.0 

73.7 

36.3 

56.6 

68.0 

73.3 

81.9 

12.0 

83.1 

30.2 

51.1 

62.5 

70.6 

83.4 

Tfcble  1 
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An  interesting  feature  of  the  dispatching  rule  is 
suggested  by  Figure  3,  obtained  from  the  Monte 
Carlo  experiments.  On  this  diagram  are  plotted  the 
inventories  of  remaining  sorties  as  functions  of  time 
for  various  actual  request  rates  when  the  rule  used 
was  calculated  from  a  forecasted  request  rate  of  6.0. 
The  rule  seems  to  have  a  built-in  ability  to  correct 
itself.  When  demands  are  very  high,  the  number  of 
sorties  available  dropped  rapidly,  causing  the  criteria 
for  dispatching  a  given  number  of  sorties  to  become 
more  stringent.  Thus,  even  when  the  request  rate  is 
badly  underestimated,  some  resources  are  still  con¬ 
served  throughout  most  of  the  horizon. 


Figure  3  — Depletion  of  aircraft  for  various  requesi  rates 
under  rule  based  on  forceasled  rale  of  6.0 


IV.  EMPIRICAL  STUDIES 
The  preceding  discussion  has  shown  how  explicit 
values  of  possible  mission  outcomes  can  be  used  to 
make  mission-dispatching  decisions.  It  has  assumed 
that  such  values  can  be  obtained.  In  a  sense,  the  truth 
of  the  assumption  is  self-evident.  A  man,  presented 
with  a  table  in  which  a  set  of  such  values  is  to  be 
entered,  can  easily  be  persuaded  to  enter  numbers. 
The  question  is  not  whether  such  numbers  can  be  ob¬ 
tained,  but  whether  they  are  appropriate  bases  for 
decision.  A  general  and  abstract  discussion  of  this 
and  some  related  questions  is  reserved  for  the  con¬ 
clusion  of  this  Memorandum 

Situations  exist  in  which  an  appropriate  external 
standard  of  value  can  be  defined.  One  necessary, 
though  far  from  sufficient,  property  that  judged  values 
should  have  if  they  are  to  be  appropriate  bases  for 
decision  is  that  they  should  in  such  situations  agree 


reasonably  well  with  the  external  standard.  A  study 
exhibiting  that  property  is  described  below. 

Another  possible  criterion  for  appropriateness  of 
judged  values  is  that  judges  should  agree.  But  this 
criterion  is  tricky.  T aken  literally,  it  denies  the  obvious 
truth  that  men  may  disagree  about  values  even  when 
they  know  the  same  facts.  Still,  a  system  whose  per¬ 
formance  depends  entirely  on  the  identity  of  its  value 
judge  is  uncomfortably  subjective  — though  perhaps 
realistically  so. 

One  way  out  of  the  dilemma  is  to  exclude  the  man 
from  the  definition  of  the  system  by  saying  that  the 
system  exists  for  the  purpose  of  implementing  its 
value  judge’s  values  as  effectively  as  possible.  If 
human  values  in  fact  differ  in  irreconcilable  ways,  no 
other  position  is  possible,  since  all  decision  systems 
must  be  based  on  human  values  in  one  way  or  another. 
But  the  facts  may  permit  a  less  subjective  resolution 
of  the  problem.  Presumably  training  should  serve  to 
create  a  communality  of  values.  If  so,  men  should 
agree  to  some  extent  on  their  value  judgments  if 
they  have  been  trained  at  all,  and  the  extent  of  that 
agreement  should  be  an  increasing  function  of  the 
amount  and  communality  of  their  training  and  ex¬ 
perience.  If  this  kind  of  agreement  is  desired,  then 
an  appropriate  performance  criterion  for  JUDGE  (or 
perhaps  for  the  larger  system,  including  selection 
and  training  procedures  for  its  value  judges,  in  which 
JUDGE  is  embedded)  would  be  that  it  enhances 
this  kind  of  agreement,  as  compared  with  its  com¬ 
petitors.  This  is,  of  eourse,  a  property  that  can  be 
studied  in  experiments  and  simulations. 

This  section  presents  the  results  of  two  studies: 
an  experiment  on  methods  for  collecting  value  judg¬ 
ments,  and  a  field  study  permitting  some  comparison 
of  JUDGE  with  current  procedures  not  based  on 
explicit  value  judgments. 

The  basic  idea  of  JUDGE  leaves  unanswered  many 
questions  about  best  system  design.  To  provide  a 
vehicle  for  answering  such  questions  (especially 
those  about  the  effect  of  operator  training  and  ex¬ 
perience  on  JUDGE’S  performance)  and  to  permit 
more  extensive  comparison  of  JUDGE  with  current 
procedures  than  was  possible  in  the  field  study,  a 
laboratory  simulation  of  a  TAC  control  system  en¬ 
vironment  is  necessary. 

The  methodology  experiment 

An  experiment  was  run  to  determine  the  best  pro¬ 
cedure  for  asking  human  subjects  for  value  judgments. 
Three  procedures  were  used  to  estimate  the  prices 
of  used  automobiles. 

I .  Ratio.  The  subject  said  how  many  times  more  or 
less  valuable  a  car  was  than  a  carefully  defined 
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standard  car,  by  placing  a  mark  on  a  scale  having 
the  digits  1  through  5  logarithmically  spaced 
on  it,  and  checking  beside  the  word  MORE  or 
LESS  to  indicate  the  direction  of  the  judgment. 

2.  Difference.  The  subject  said  how  much  more  or 
less  valuable  a  car  was  than  the  standard  car, 
by  placing  a  mark  on  a  scale  the  same  size  as 
the  one  for  ratio  judgments,  but  with  figures  of 
$500  to  $2000  linearly  spaced  on  it,  and  marking 
MORE  or  LESS  to  indicate  the  direction  of  the 
judgment.  Both  this  scale  and  the  ratio  one  were 
open  ended  at  the  top.  The  standard  car  was  the 
same  one  used  in  the  ratio  procedure. 

3.  Direct.  The  subject  simply  estimated  the  value 
of  the  car  he  was  judging. 

The  stimulus  material  consisted  of  four  equivalent 
sets  of  10  automobiles,  ranging  in  retail  price  from 
about  $500  to  about  $4500  in  the  Kelly  Blue  Book , 
and  described  as  they  would  be  in  an  advertisement 
for  used  cars.  One  car,  priced  at  $1800,  was  included 
on  all  four  lists,  and  was  used  as  the  standard  in  the 
two  procedures  that  required  one.  A  fourth  task,  that 
of  paired  comparisons,  was  administered  to  the  sub¬ 
jects,  but  the  results  from  this  task  are  not  reported 
here.  (The  laboriousness  of  paired  comparisons 
precludes  their  use  in  systems  requiring  numerous 
human  judgments.) 

The  subjects,  40  volunteer  college  students  who 
were  paid  for  their  two  hours  of  participation  in  the 
experiment,  were  randomly  divided  into  four  groups. 
Each  group  received  a  different  pairing  of  the  four 
procedures  with  the  four  groups  of  cars.  Thus  each 
subject  used  all  procedures  and  judged  all  cars,  mak¬ 
ing  a  total  of  40  judgments.  The  combination  of 
groups  of  subjects,  procedures,  list  of  cars,  and  order 
of  presentation  was  specified  by  a  Greco-Latin  square 
design  so  that  all  order  effects  and  effects  of  combina¬ 
tions  of  procedures  and  lists  of  cars  would  be  com¬ 
pletely  counterbalanced. 

The  subjects’  responses  were  transformed  into  dol¬ 
lar  values.  For  the  ratio  procedure,  this  was  done  by 
multiplying  the  Blue  Book  value  of  the  standard  by 
the  subject’s  judgment  for  the  other  car.  For  the  dif¬ 
ference  procedure,  the  subject’s  judgment  for  the 
other  car  was  added  to  or  subtracted  from  the  Blue 
Book  value  of  the  standard.  For  the  direct  estima¬ 
tion  procedure,  the  ratio  of  the  subject’s  estimates 
for  the  other  car  and  for  the  standard  car  was  cal¬ 
culated,  and  then  processed  by  the  same  procedure 
used  for  the  ratio  scale  estimates.  (This  served  to 
adjust  the  direct  estimates  for  a  form  of  individual 
bias.)  The  performance  measure  was  the  mean  square 
error  for  each  subject  on  the  nine  cars  he  judged  by 


each  procedure,  the  error  being  the  difference  be¬ 
tween  the  estimated  dollar  value  and  the  “true” 
Blue  Book  value. 

An  analysis  of  variance  was  performed  on  these 
error  scores.  The  Greco-Latin  square  design  of  course 
precludes  calculation  of  variance  estimates  for  inter¬ 
actions.  None  of  the  methodological  variables  (groups 
of  subjects,  lists  of  cars,  and  order  of  presentation) 
made  any  systematic  difference  to  error.  The  effect 
of  scaling  procedure  was  highly  significant;  inspection 
of  mean  confirms  this  conclusion.  The  mean  squared 
error  (times  10~5)  for  the  ratio  procedure  was  31.86; 
for  the  difference  procedure,  6.62;  and  for  the  direct 
estimation  procedure,  8.37. 

On  the  basis  of  these  results,  the  difference  pro¬ 
cedure  was  chosen  for  use  in  the  Field  study  described 
below.  Although  it  and  the  direct  procedure  are 
equally  attractive  on  the  basis  of  these  data,  the 
availability  of  a  natural  scale  of  value  (that  is,  dollars) 
for  used  cars  makes  the  direct  procedure  more  ap¬ 
propriate  for  them  than  it  would  be  in  general. 

To  estimate  the  overall  reliability  of  the  groups  of 
subjects,  we  calculated  the  product-moment  correla¬ 
tion  coefficients  between  the  values  obtained  by  each 
procedure  and  the  Blue  Book  values.  Table  3  shows 
the  means  over  subjects  within  procedures  and  the 
standard  deviations  for  the  three  procedures.  Note 
that  the  correlations  are  gratifyingly  high.  This  means 
not  only  that  the  subjects  agreed  well  with  the  Blue 
Book ,  but  also  that  they  agreed  well  with  one  another. 

As  a  further  check  on  intersubject  agreement,  we 
calculated  product-moment  correlations  between  all 
possible  pairs  of  subjects  within  each  group  of  ten 
subjects.  The  means  and  standard  deyiations  of  these 
correlations  are  shown  in  Table  4.  The  size  of  these 
numbers  is  evidence  of  the  high  reliability  of  the  esti¬ 
mation  process. 

Table  3  — Mean  product-moment  correlation 
coefficients  between  estimated  and  actual 
values  over  subjects  within  procedures 


Ratio 

Direct 

Difference 

M 

(T 

M 

a 

M 

<T 

.894 

.066 

.810 

.074 

.819 

.084 

.787 

.116 

.900 

.046 

.729 

.150 

.822 

.105 

.886 

.132 

.919 

.051 

.887 

.078 

.927 

.135 

.926 

.051 
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Table  4  — Means  and  standard  deviations  for 
product  moment  correlation  coefficients  between 
all  subject  pairs  within  each  group  of  ten  subjects 


Ratio 

Direct 

Difference 

M 

CT 

M 

CT 

M 

CT 

.779 

114 

.800 

.113 

.789 

.115 

.709 

.157 

.849 

.086 

.712 

.130 

.665 

.192 

.769 

.166 

.887 

.063 

.824 

.090 

.925 

.043 

.876 

.062 

The  field  study 

The  field  study  was  performed  to  explore  tech¬ 
niques  for  obtaining  value  judgments  in  relatively 
realistic  situations,  and  to  examine  JUDGE’S  per¬ 
formance  based  on  those  value  judgments.  The  exer¬ 
cise,  carried  out  in  a  classroom-like  situation,  used 
scenario  materials  based  on  an  unclassified  lesson 
plan  developed  at  the  U.S.  Army  Command  and  Gen¬ 
eral  Staff  College  at  Fort  Levenworth,  Kansas.  Seven¬ 
teen  Air  Force  officers  familiar  with  tactical  air  war¬ 
fare  served  as  subjects.  These  were  a  group  of  For¬ 
ward  Air  Controllers  and  Air  Liaison  Officers  sta¬ 
tioned  at  Cannon  Air  Force  Base  in  Clovis,  New 
Mexico. 

The  subjects  were  given  a  brief  description  of  the 
political  situation  leading  to  the  hypothetical  conflict, 
and  a  detailed  description  of  the  battle  situation  was 
given  using  a  large  scale  map.  The  battle  was  set  in 
Southeast  Asia,  and  involved  three  divisions  of  allied 
forces  operating  against  a  roughly  comparable  enemy 
force.  The  subjects  completed  the  entire  experiment 
described  in  the  remainder  of  this  paragraph  in  a 
single  three-hour  session.  The  experiment  was  con¬ 
cerned  with  two  successive  days  of  a  battle,  and  two 
hours  of  each  of  these  days  were  simulated.  Each  two- 
hour  period  is  referred  to  as  a  situation  in  the  ex¬ 
perimental  design.  Additional  narrative  material  pro¬ 
vided  the  transition  from  the  first  to  the  second  day. 

During  each  simulated  two-hour  period,  a  sequence 
of  targets  was  presented,  one  at  a  time.  A  target  re¬ 
port  was  a  printed  form  giving  a  description  of  the 
target,  its  location,  the  source  of  the  report;  for  the 
DASC*  system,  a  mission  effectiveness  function  and 
the  time  of  the  report  were  also  included.  A  sample 
target  report  for  the  DASC  system  appears  as  Figure 
4.  For  the  DASC  portions  of  the  experiment,  subjects 
were  told  that  the  mission  effectiveness  function  as¬ 


*DASC  is  the  aulhors*  abstraction  of  the  current  operating  system. 


sociated  with  a  particular  target  embodied  all  relevant 
information  concerning  aircraft  and  ordnance  per¬ 
formance.  The  graph  of  a  mission  effectiveness  func¬ 
tion  indicates  the  probability  that  the  target  will  be 
destroyed  or,  in  the  case  of  a  distributed  target,  the 
portion  of  the  target  expected  to  be  destroyed  as  a 
function  of  the  number  of  aircraft  dispatched. 


TAC  TARGET 
CLOSE  AIR  SUWORT 


T«r»*t  No. 

Coordlnot**: 

Location  i 

A  -  0 

T  1019 

R*po <t*d  by*. 

Tim: 

Description 


2nd  »nd  3rd  Me  .  edvenced  e  1  even 


t* 


0900 


Advanced  e)e»enta  under  etteck  by  inny  tank 
platoon.  Attack  con  elate  of  five  tanka,  typa 
unknown,  with  105m  guna  and  haavy  B.g.'e  Mounted 


MISSION  EFFECTIVENESS  TA11E 


Figure  4  — Target  report  for  DASC 

To  compare  JUDGE  with  sortie  allocations  made 
in  the  usual  way  (the  DASC  system),  the  subjects 
were  exposed  to  each  battle  situation  twice.  Acting 
as  decision  makers  in  the  DASC  system,  the  subjects 
allocated  specific  numbers  of  sorties  against  the 
targets  as  they  appeared;  acting  as  value  estimators 
in  the  JUDGE  system,  they  assigned  values  to  the 
targets  as  they  appeared.  The  subjects  were  instructed 
to  try  not  to  let  their  responses  under  one  operating 
mode  affect  their  decisions  under  the  other.  Two 
groups  of  subjects  were  formed  and  the  ordering  of 
systems  was  counterbalanced  as  shown  below. 


Situation  1 

Situation  2 

Order 

i 

2 

3 

4 

Group  1 
Group  2 

DASC 

JUDGE 

DASC 

JUDGE 

DASC 

JUDGE 

DASC 

JUDGE 
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Before  operating  in  the  DASC  mode,  the  subjects 
were  told  that  they  had  40  sorties  to  dispatch  in  the 
two-hour  situation  and  that  they  should  expect  to 
receive  requests  at  the  rate  of  about  ten  per  simulated 
hour.  (The  actual  times  on  the  target  reports  were 
selected  by  a  random  process  which  yielded  18  targets 
for  the  first  situation  and  22  targets  for  the  second.) 
Six  different  mission  effectiveness  functions  of  the 
form  rj(\)  =  1  —  (1  —  p)x°were  used  and  were  dis¬ 
played  to  the  subjects  during  the  instructional  period. 
The  subjects  were  told  that  the  distribution  of  ef¬ 
fectiveness  functions  over  targets  would  be  uniform. 

For  the  JUDGE  task,  the  target  described  in  Figure 
4  was  used  as  a  standard.  The  subjects  were  to  as¬ 
sociate  a  value  of  100  with  this  target  and  to  consider 
an  utterly  worthless  target  as  having  a  value  of  0. 
All  other  targets  were  to  be  valued  relative  to  these 
fixed  numbers.  The  value  to  be  judged  was  the  im¬ 
portance  of  destroying  the  target. 

To  convert  the  subjects’  value  responses  into  dis¬ 
patch  decisions,  a  computer  program  was  written  to 
calculate  the  JUDGE  dispatching  rule.  This  program 
treats  the  prior  distributions  of  value  judgments  and 
mission  effectiveness  functions  as  independent.  In 
testing  the  JUDGE  system,  two  different  value  dis¬ 
tributions  have  been  employed.  Distribution  U  is  a 
uniform  (rectangular)  distribution  with  a  range  of 
values  between  0  and  150;  distribution  T  is  triangular 
with  a  mode  of  0  and  gives  nonzero  probabilities  for 
values  up  to  225.  Both  of  these  one-parameter  dis¬ 
tributions  have  means  of  75,  their  major  character¬ 
istics  are  shown  in  Table  5.  Distribution  U  was 
chosen  because  of  its  simplicity,  while  distribution 
T  was  suggested  by  observing  a  histogram  of  all 
value  responses  obtained  in  the  field  study.  This 
histogram  was  constructed  with  very  broad  intervals 
to  smooth  over  subject  preferences  for  certain  round 
numbers. 

Table  5  — Prior  distributions  of  values 
used  for  JUDGE 


Distribution 

U  (Uniform) 

T  (Triangular 
with  mode  0) 

Density  Y  unction 

-  for  (0  v  «=  c) 

'T 1  —  -)  for  (0  ^  v  ^  c) 

c 

c  c 

Mean 

!-« 

Standard  Deviation 

—  =  43.3 

=  53.0 

2V3 

3V2 

As  a  prior  estimate  for  the  distribution  of  mission 
effectiveness  functions  over  targets  in  each  situation, 
we  used  the  actual  distribution  taken  over  both  situa¬ 
tions.  This  distribution  assigned  nearly  equal  prob¬ 
abilities  to  the  six  functions  and  was  not  exactly  cor¬ 
rect  for  either  situation  1  or  2.  In  addition,  the  pro¬ 
gram  was  supplied  with  the  forecasted  rate  of  10  re¬ 
quests  per  hour,  the  horizon  length  of  two  hours, 
and  the  initial  availability  of  40  sorties. 

For  each  simulated  time  point,  t,  at  which  a  target 
was  presented  during  the_DASC  phases  of  experi¬ 
ment,  a  vector  of  values,  Wm(t),  (m  =  0,  2, . . .  ,40)  was 
stored.  Given  a  subject’s  sequence  of  value  judg¬ 
ments,  the  corresponding  JUDGE  dispatches  can 
then  be  easily  calculated.  In  both  systems,  missions 
were  required  to  have  multiples  of  two  aircraft. 

The  means  and  standard  deviations  of  the  subjects’ 
value  responses  are  tabulated  in  Table  6.  Comparing 
these  data  with  the  information  in  Table  5  indicates 
that  both  of  our  prior  value  distributions  were  in  con¬ 
siderable  error  for  many  of  the  subjects.  To  be  useful, 
however,  JUDGE  must  be  robust  against  forecast 


errors. 


Table  6 — 

Means  and  standard  deviations  of  value  judgments 


Subj 

Situation  1 

Situation  2 

Mean 

Std-Dev 

Mean 

Std-Dev 

1 

55.8 

47.7 

58.3 

45.5 

2 

120.0 

51.6 

122.9 

58.4 

3 

68.1 

58.8 

92.5 

68.8 

4 

67.8 

63.1 

106.9 

82.2 

5 

85.0 

41.8 

84.6 

59.4 

13 

45.6 

45.0 

53.8 

59.9 

14 

47.5 

49.2 

58.1 

48.8 

15 

82.8 

52.6 

96.0 

66.1 

16 

75.2 

36.0 

90.4 

23.5 

17 

71.7 

58.5 

72.5 

55.7 

Mean 

72.0 

50.4 

83.6 

56.8 

Std-Dev 

20.5 

7.9 

21.6 

14.8 

6 

124.2 

54.1 

120.2 

40.6 

7 

106.9 

44.7 

95.0 

30.0 

8 

65.6 

27.7 

69.8 

36.3 

9 

96.7 

54.4 

60.6 

29.4 

10 

68.6 

23.3 

80.6 

35.1 

11 

91.1 

32.3 

76.2 

41.9 

12 

57.5 

51.9 

53.1 

39.6 

Mean 

87.2 

41.2 

79.3 

36.1 

Std-Dev 

22.5 

12.2 

20.9 

4.6 

Grand 

Mean 

78.2 

46.6 

81.8 

48.3 

Std-Dev 

22.6 

10.9 

21.4 

15.5 

Group 
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Comparison  of  JUDGE  and  DASC 

To  provide  a  basis  for  a  meaningful  comparison 
of  JUDGE  and  DASC,  we  examined  two  other  dis¬ 
patching  rules.  One  was  a  theoretical  optimum,  the 
best  that  could  have  been  done  with  perfect  fore¬ 
knowledge  of  the  incidence  and  value  of  all  requests 
during  the  two-hour  period.  In  the  absence  of  such 
perfect  foreknowledge,  this  optimum  is  of  course 
unattainable.  To  calculate  the  perfect  sequence  of 


dispatching  decisions,  pairs  of  planes  were  assigned 
to  targets  in  order  of  decreasing  marginal  utility,  ignor¬ 
ing  the  sequence  in  which  the  targets  arrived,  until 
all  planes  were  used  up.  This  optimum  differs  from 
one  subject  to  another,  since  it  is  based  on  value  judg¬ 
ments. 

The  other  dispatching  rule  was  first-come,  first- 
served  (FCFS).  We  recognized  that  any  dispatching 
rule,  no  matter  how  absurd,  would  be  certain  to  ob- 


Tablc  7— 

SYSTEM  PERFORMANCE:  SITUATION  1 


Expected  Utility 

Effectiveness 

Subj 

FCFS 

DASC 

JUDGE(U) 

JUDGE(T) 

Optimum 

DASC 

JUDGE(U) 

JUDGE(T) 

1 

238 

395 

450 

447 

480 

0.65 

0.88 

0.86 

2 

351 

419 

532 

548 

619 

0.26 

0.68 

0.74 

3 

251 

333 

449 

445 

459 

0.40 

0.95 

0.94 

4 

193 

297 

398 

394 

417 

0.46 

0.92 

0.90 

5 

341 

435 

531 

530 

530 

0.50 

1.00 

1.00 

6 

329 

730 

772 

779 

850 

0.77 

0.85 

0.86 

7 

470 

651 

754 

774 

111 

0.59 

0.92 

0.99 

8 

246 

352 

439 

439 

441 

0.54 

0.99 

0.99 

9 

374 

545 

662 

652 

674 

0.57 

0.96 

0.93 

10 

224 

325 

404 

395 

433 

0.48 

0.86 

0.82 

1  1 

342 

471 

578 

579 

594 

0.52 

0.94 

0.94 

12 

177 

249 

404 

408 

420 

0.30 

0.94 

0.95 

13 

171 

159 

346 

342 

375 

0.06 

0.86 

0.97 

14 

310 

472 

465 

462 

497 

0.86 

0.83 

0.81 

15 

249 

329 

466 

451 

496 

0.33 

0.88 

0.82 

16 

290 

377 

467 

464 

499 

0.42 

0.85 

0.83 

17 

368 

376 

509 

512 

543 

0.05 

0.81 

0.82 

Mean  0.448  0.888  0.893 

Table  8— 


SYSTEM  PERFORMANCE:  SITUATION  2 


Subj 

1 

Expected  Utility 

Effectiveness 

FCFS 

DASC 

JUDGE(U) 

JUDGE(T) 

Optimum 

DASC 

JUDGE(U) 

JUDGE(T) 

328 

410 

436 

461 

467 

0.59 

0.78 

0.96 

2 

514 

629 

666 

708 

769 

0.45 

0.60 

0.76 

3 

362 

493 

539 

550 

563 

0.65 

0.88 

0.94 

4 

410 

536 

592 

605 

629 

0.58 

0.83 

0.89 

5 

374 

442 

563 

562 

603 

0.30 

0.82 

0.82 

6 

492 

658 

769 

783 

841 

0.48 

0.80 

0.84 

7 

389 

581 

683 

694 

695 

0.63 

0.96 

1.00 

8 

310 

456 

510 

524 

536 

0.64 

0.86 

0.95 

9 

275 

331 

411 

408 

424 

0.38 

0.91 

0.89 

10 

313 

510 

552 

575 

575 

0.75 

0.91 

1.00 

11 

282 

323 

488 

479 

488 

0.20 

1.00 

0.96 

12 

176 

250 

296 

282 

296 

0.62 

1.00 

0.88 

13 

292 

325 

427 

461 

461 

0.20 

0.80 

1.00 

14 

241 

496 

499 

489 

536 

0.86 

0.88 

0.84 

15 

383 

454 

551 

580 

626 

0.29 

0.69 

0.81 

16 

432 

488 

603 

599 

616 

0.30 

0.93 

0.90 

17 

247 

494 

555 

559 

567 

0.77 

0.96 

1.00 

Mean 


0.511 


0.861 


0.908 
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tain  some  expected  utility  if  it  dispatched  aircraft  at 
all.  Only  improvements  over  some  minimal  per¬ 
formance  level  should  be  given  credit.  So  to  specify 
such  a  minimal  system,  we  chose  to  dispatch  planes 
four  at  a  time  to  all  targets  until  planes  were  ex¬ 
hausted. 

Table  7  shows  the  results  for  situation  1,  and  Table 
8  for  situation  2.  The  first  five  columns  contain  the 
expected  utilities  earned  by  FCFS,  DASC,  JUDGE 
(U),  JUDGE(T),  and  the  optimum  system  for  each 
subject,  based  on  each  individual’s  value  judgments. 
The  column  headings  JUDGE(U)  and  JUDGE(T) 
are,  respectively,  JUDGE  computed  with  the  U  and  T 
value  distributions  described  in  Table  5.  The  last 
three  columns  contain  relative  effectiveness  numbers 
for  DASC  and  the  two  versions  of  JUDGE.  For 
each  subject,  they  are  calculated  by  treating  the  ex¬ 
pected  utility  obtained  by  FCFS  as  0  and  the  expected 
utility  obtained  by  the  perfect  system  as  I.  With  this 
definition  of  the  origin  and  a  unit  of  measurement 
of  the  utility  function,  DASC  performs  about  50  per 
cent  as  well  as  the  unattainable  optimum,  while 
JUDGE  performs  about  90  per  cent  as  well  as  the 
perfect  system. 

The  large  discrepancy  between  JUDGE  and  DASC 
indicates,  as  expected,  that  JUDGE  is  much  more 
efficient  in  implementing  a  subject’s  values  than  the 
subject  is  himself.  JUDGE  separates  the  evaluation 
portion  of  the  dispatching  task  (the  portion  that  de¬ 
pends  on  human  expertness)  from  the  decision  making 
portion,  which,  given  the  value  judgments,  is  a  dif¬ 
ficult  computational  task  that  a  computer  can  per¬ 
form  more  effectively  than  a  man. 

It  is  important  that  DASC,  while  much  inferior  to 
JUDGE,  is  more  superior  to  FCFS.  Since  the  scores 
justifying  this  assertion  are  based  on  the  value  judg¬ 
ments,  this  finding  means  that  the  DASC  decisions 
are  by  no  means  unrelated  to  the  JUDGE  value  judg¬ 
ments.  It  is  reasonable  to  believe  that  each  subject’s 
JUDGE  dispatches  and  his  DASC  dispatches  arc 
attempts  to  implement  the  same  set  of  values;  the 
JUDGE  dispatches  are  simply  more  effective  at  doing 
so. 

An  incidental  observation  makes  the  same  point. 
One  of  the  authors,  familiar  with  the  stimuli  ahead 
of  time,  and  with  complete  knowledge  of  the  scoring 
rules,  performed  the  role  of  a  subject  twice.  Though 
he  attempted  to  make  DASC  dispatches  that  would 
produce  high  scores,  his  data  closely  resembled  that 
reported  in  Tables  7  and  8.  It  is  simply  difficult  to 
translate  a  value  system  into  dispatching  decisions, 
and  JUDGE  does  it  much  better  than  men  can. 

That  JUDGE(U)  and  JUDGE(T)  are  so  nearly 
alike  further  indicates  that  JUDGE  is  rather  robust 


under  variation  in  the  distribution  of  value  judgments 
from  its  prior  expectation. 

We  attempted  to  scale  the  subjects’  situation  2 
responses  based  on  their  situation  1  value  judgments 
to  see  if  the  effectiveness  of  JUDGE  could  be  im¬ 
proved  by  making  a  subject’s  distribution  of  value 
judgments  correspond  more  closely  to  the  forecasted 
distribution.  Since  there  was  a  considerable  degree 
of  correlation  between  the  subject’s  means  and 
standard  deviations  between  the  two  situations,  it 
is  reasonable  to  suppose  that  it  would  be  profitable 
to  modify  a  subject’s  responses  using  statistics  ob¬ 
tained  from  observing  him  at  an  earlier  time.  The 
mean  and  standard  deviation  of  distribution  U  are 
75  and  43.3  respectively.  Suppose  that  a  particular 
subject’s  mean  and  standard  deviation  in  situation  I 
were  m  and  s.  Letting  x  represent  a  response  in  situa¬ 
tion  2,  an  adjusted  response  for  that  target  was  com¬ 
puted  by  the  formula 


43.3  , 

xadi  =  -7—  (x  -  m)  +  75. 


This  simple  adjustment  is  just  a  stretching  (or  com¬ 
pressing)  and  a  movement  of  the  origin  of  the  subject’s 
response  scale  based  on  his  previous  responses. 

The  application  of  this  adjustment  technique  re¬ 
sulted  in  very  modest  effectiveness  increases  for 
JUDGE(U).  That  the  improvement  was  only  slight 
was  to  be  expected  from  the  already  high  effective¬ 
ness  of  JUDGE  and  from  the  large  amount  of  robust¬ 
ness  already  evident. 

Intercorrelations  among  subject’s  scores 

So  far,  evidence  indicates  that  JUDGE  implements 
each  subject’s  values  better  than  his  own  decisions 
can.  Ideally,  we  would  like  to  show  the  relation  be¬ 
tween  an  individual  subject’s  value  judgments  and 
some  ultimate  criterion  of  value,  such  as  “winning  the 
war.’’  Unfortunately,  we  do  not  know  how  to  derive 
target  values  from  any  such  ultimate  criterion.  In¬ 
deed,  if  we  could  do  so,  such  calculated  values  rather 
than  human  judgments  should  be  the  values  that 
JUDGE  translates  into  decisions. 

Thus,  no  ultimate  validation  of  JUDGE,  or  of  any 
command  system,  is  possible.  The  philosophical 
basis  for  this  conclusion  is  the  subject  of  the  last 
section  of  this  Memorandum.  However,  relevant  ques¬ 
tions  can  be  examined.  For  example,  intra-subject 
and  intcr-subject  reliability  are  both  relevant.  If  a 
subject’s  value  judgments  collected  at  one  time  sys¬ 
tematically  differ  from  his  value  judgments  for  the 
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same  targets  in  the  same  situation  collected  at  a  dif¬ 
ferent  time,  there  would  be  some  doubt  about  the 
appropriateness  of  implementing  either  set  of  values. 
Unfortunately,  the  field  study  provides  no  information 
about  intra-subject  reliability  since  there  was  no  repli¬ 
cation.  However,  the  use  of  many  subjects  permits 
us  to  examine  the  extent  to  which  one  subject  agrees 
with  another. 

Table  9  presents  the  mean  intercorrelations  among 
subjects  within  groups  and  situations  for  three  sets 
of  numbers:  DASC  dispatching  decisions,  value 
judgments,  and  JUDGE  dispatching  decisions  based 
on  those  judgments.  Three  points  can  be  noted  about 
these  numbers.  First,  they  are  all  low.  Clearly  these 
subjects  disagreed  with  one  another  both  about  how 
valuable  the  targets  were  and  about  how  many  planes 
to  send  against  each.  Second,  in  every  case,  the  mean 
intercorrelation  between  JUDGE  dispatching  de¬ 
cisions  is  higher  than  that  for  the  DASC  decisions 
and  higher  than  that  for  the  value  judgments  on  which 
they  are  based.  These  decisions  reflect  not  only  those 
values,  which  differ  from  subject  to  subject,  but  also 
mission  effectiveness  functions  and  times  at  which 
the  requests  arrived,  both  constant  across  subjects. 
It  is  gratifying  that  the  JUDGE  intercorrelations 
are  higher  than  those  for  DASC.  Perhaps  the  most 
suggestive  feature  of  Table  9,  however,  is  that  the 
mean  intercorrelations  for  situation  2  are  higher  in 
all  cases  than  those  for  situation  1,  and  JUDGE  gains 
more  than  DASC.  Clearly  learning  is  going  on,  and  it 
seems  unlikely  that  it  has  reached  its  limits. 

Tabic  9— 


Average  correlations  of  subjects  with  all  other  subjects 


Situation 

1 

Situation  2 

DASC 

JUDGE 

JUDGE 

DASC 

JUDGE 

JUDGE 

Group 

Dis¬ 

patches 

Values 

Dis¬ 

patches 

Dis¬ 

patches 

Values 

Dis¬ 

patches 

1 

0.346 

0.413 

0.461 

0,396 

0.539 

0.550 

2 

0.252 

0.238 

0.358 

0.396 

0  530 

0  566 

Further  experimentation  in  a  laboratory  where 
greater  demands  on  a  subject’s  time  can  be  made  is 
necessary  to  establish  the  upper  limits  of  learning 
and  both  kinds  of  reliability.  The  field  study  has  es¬ 
tablished  that  JUDGE  works;  it  is  much  superior  to 
DASC,  and  it  is  robust  against  various  kinds  of  devia¬ 
tions  from  prior  expectation. 

V.  GENERAL  COMMENTS  ON  VALUE- 
JUDGMENT  BASED  COMMAND  SYSTEMS 
As  we  worked  on  JUDGE,  we  found  ourselves 
strongly  influenced  by  a  set  of  philosophical  ideas 
about  judgment-based  systems  in  general.  These  ideas 


have  to  do  with  the  purpose,  design,  and  evaluation 
of  such  systems.  Most  of  our  ideas  about  the  TAC 
problem  follow  from  these  more  general  considera¬ 
tions— though  many  of  them  could  be  justified  from 
other,  less  radical,  points  of  view  than  the  one  we 
present  here.  Since  we  have  seen  no  other  presenta¬ 
tion  of  the  position  we  have  come  to,  and  indeed  very 
little  discussion  of  the  issues  underlying  validation 
of  command  systems  exists  on  paper,  we  have  chosen 
to  end  this  Memorandum  with  a  fairly  extended  dis¬ 
cussion  of  these  philosophical  questions. 

We  believe  that  all  command  systems  are  and  must 
always  be  judgment-based.  A  command  system  exists 
to  make  decisions.  We  find  it  useful  to  distinguish 
between  the  decision,  or  selection  of  an  action,  and 
the  decision  process.  A  decision  process  describes  a 
complex  sequence  of  events  beginning  with  recogni¬ 
tion  that  an  action  will  have  to  be  selected,  and  end¬ 
ing  with  implementation  of  the  selected  action.  We 
believe  that  human  judgments  play  an  absolutely 
necessary  role  in  all  decision  processes,  at  least  for 
decisions  important  and  interesting  enough  to  concern 
command  systems.  Our  goal  is  to  analyze  the  decision 
process  into  functions,  to  ascertain  how  and  by  what 
means  each  function  should  be  performed,  and  to 
allocate  those  functions  best  performed  by  men  to 
men,  and  those  best  performed  by  machines  to  ma¬ 
chines.  The  assertion  that  all  command  systems  are 
and  must  always  be  judgment-based  means,  then,  that 
the  set  of  functions  best  performed  by  men  will  never 
turn  out  to  be  empty.  One  reason  for  that  assertion 
is  that  command  systems  exist  to  serve  human  pur¬ 
poses,  and  require  men  to  specify  what  those  pur¬ 
poses  are.  We  believe  also  that,  at  least  for  a  long 
time  to  come,  men  will  be  indispensable  for  a  number 
of  other  functions  in  the  decision  process,  as  the  fol¬ 
lowing  discussion  exhibits. 

The  fact  that  all  command  systems  are  judgment- 
based  is  often  not  explicitly  recognized  because  of  a 
faulty  definition  of  system  boundaries.  The  statement 
is  sometimes  made,  for  instance,  that  “the  function  of 
a  command  system  is  to  help  the  commander  do  his 
job.”  This  is  too  narrow  a  definition  of  the  system. 
In  the  definition  that  we  consider  appropriate,  the 
commander  is  a  part  of  the  system,  and  the  function 
of  the  entire  system,  commander  and  all,  is  to  make 
“the  right  decisions.” 

When  is  a  decision  right?  Everyday  evaluation  of 
decisions  is  usually  done  by  comparing  the  choice 
actually  made  with  the  one  that  would  have  been 
made  either  by  the  man  doing  the  comparing  or  by 
some  appropriate  authority.  If  the  person  making  the 
comparison  is  the  one  who  made  the  decision,  then 
most  decisions  are  right  by  definition,  and  this  defini- 
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tion  of  rightness  is  uninteresting.  If  someone  else 
judges  whether  a  decision  is  right  or  wrong,  then  when 
he  says  it  is  wrong  we  have  two  experts  who  dis¬ 
agree,  and  usually  no  satisfactory  means  of  resolving 
the  disagreement. 

The  search  for  a  way  out  of  this  impasse  has  char¬ 
acterized  almost  all  research  on  command  systems. 
We  feel  that  a  departure  is  possible  only  after  radical 
redefinition  of  what  is  meant  by  “right”  in  this  con¬ 
text.  The  remainder  of  this  discussion  presents  our 
redefinition  and  shows  how  to  use  it. 

Validity  and  reliability 

The  notion  that  there  is  a  right  action,  answer, 
diagnosis,  or  other  system  output,  and  that  the  prob¬ 
lem  of  system  design  is  to  devise  a  system  that  pro¬ 
duces  this  right  output  reminds  us  of  the  problem  of 
validating  the  design  of  intelligence  tests  or  other 
personnel  selection  instruments.  The  test  designer 
may  start  out  with  the  rather  simple  idea  that  some 
abstract  quantity  like,  say,  intelligence  exists  and  that 
his  task  is  to  measure  that  quantity  in  the  individuals 
he  tests.  He  develops  a  possible  method  for  measure¬ 
ment,  and  then  must  consider  whether  or  not  it  really 
measures  what  he  hopes  it  measures.  How  can  he  find 
out? 

Two  main  approaches  have  been  taken  to  the  prob¬ 
lem  of  validating  tests.  One,  the  criterion-oriented 
approach,  depends  on  comparing  the  test  under  study 
with  some  other,  already  available  measure  of  the 
quantity  to  be  measured.  If  test  A  is  “known”  to 
measure  intelligence,  then  test  B  can  be  considered 
valid  if  it  correlates  highly  with  test  A.  In  the  case 
of  intelligence  tests,  Binet’s  original  idea  was  to  use 
success  in  school  as  his  validating  criterion. 

Criterion-oriented  validity  has  two  major  diffi¬ 
culties.  One  is  that  most  criteria  are  themselves  sus¬ 
pect,  either  because  they  are  of  doubtful  relevance  to 
the  abstract  entity  to  be  measured  (is  success  in  school 
really  primarily  a  function  of  intelligence?)  or  because 
as  measures  the  criteria  themselves  have  unattractive 
properties,  such  as  unreliability,  or  both.  The  other  is 
that  for  most  abstract  entities  we  might  want  to  test 
(e.g.  propensity  to  take  risks)  no  appropriate  criterion 
is  available. 

Because  of  these  difficulties,  the  testers  have  de¬ 
veloped  a  second  and  quite  different  approach  to 
validity,  which  they  call  construct  validity.  (There 
is  yet  another  concept  called  content  validity,  having 
to  do  with  validating  tests  of  mastery  of  a  subject 
matter,  but  it  can  be  neglected  here.)  The  basic  idea 
of  construct  validity  is  that  a  test  should  make  sense 
and  data  obtained  by  means  of  it  should  make  sense. 


One  form  of  making  sense  is  that  different  proce¬ 
dures  that  purport  to  measure  the  same  abstract 
quantity  should  co-vary.  If  one  procedure  is  taken  as 
a  criterion  for  the  other,  this  is  simply  criterion- 
oriented  validity.  But  even  if  neither  procedure  is 
taken  as  valid,  a  priori,  the  fact  that  they  correlate 
highly  indicates  that  to  some  extent  they  measure  the 
same  thing  or  closely  related  things.  If,  in  addition, 
the  kind  of  underlying  quantity  that  each  might  tap 
seems  on  some  a  priori  basis  to  be  the  same,  then 
observation  of  covariation  is  an  instance  of  what  has 
been  called  converging  operations,  and  contributes  to 
validity.  (The  concept  of  construct  validity  is  broader 
than  this  description  indicates;  this  discussion  serves 
only  to  give  it  flavor.) 

A  procedure  (test,  system,  etc.),  whether  or  not 
it  is  valid,  should  certainly  be  reliable.  This  word, 
taken  from  test  theory,  is  a  bit  too  specific  for  our 
purpose.  We  will  say  that  a  procedure  should  be  in¬ 
tellectually  coherent.  One  requirement  of  intellectual 
coherence  is  that  repeated  attempts  to  measure  the 
same  thing,  or  equivalent  things,  by  means  of  the 
procedure  should  produce  the  same  measurements. 
Another  requirement  of  coherence  is  that  variables 
that  seem  irrelevant  to  the  procedure  should  not  af¬ 
fect  it. 

Other  requirements  of  coherence  exist.  For  de¬ 
cision-making  procedures,  logical  consistency  is  one 
of  them.  Thus  if  a  decision-making  system  prefers 
act  A  to  act  B  and  act  B  to  act  C,  it  should  not  prefer 
act  C  to  act  A;  it  should  be  transitive.  Similarly,  it 
should  exhibit  the  properties  known  in  decision  theory 
as  avoidance  of  dominated  strategies,  independence 
from  irrelevant  alternatives,  and  a  few  others. 

A  final  class  of  coherence  requirements  is  harder 
to  describe.  There  are  obvious  expectations  about  the 
behaviors  of  certain  systems.  An  information  process¬ 
ing  system,  for  example,  should  not  act  as  though  its 
odds  for  hypothesis  A  have  been  increased  as  a  result 
of  evidence  that  clearly  makes  A  less  likely  than  it 
was  before.  Information  that  an  act  has  become  more 
valuable  than  it  was  before  should  not  cause  a  de¬ 
cision-making  system  to  become  less  likely  than  it 
was  before  to  choose  that  act. 

Now  that  the  concept  of  coherence  has  been  intro¬ 
duced,  it  becomes  easier  to  talk  about  validity.  Valida¬ 
tion  is  simply  establishing  the  coherence  of  a  pro¬ 
cedure,  or  several  procedures.  Thus  no  sharp  line 
separates  the  concept  of  reliability  from  that  of 
validity;  both  concepts  refer  to  agreements  among 
measures,  and  a  continuum  exists  from  cases  in  which 
the  measures  essentially  repeat  the  same  procedure 
(reliability)  to  cases  in  which  rather  different  pro¬ 
cedures  seem  to  measure  the  same  thing  (validity). 
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The  validation-type  reliability  of 

judgmental  systems 

We  assert  that  no  external  measure  of  the  per¬ 
formance  of  a  judgment-based  decision-making  sys¬ 
tem  is  possible.  Any  such  measure  would  have  to 
compare  the  decisions  the  system  made  with  de¬ 
cisions  made  some  other  way,  and  there  would  have 
to  be  some  good  reason  to  suppose  that  the  decisions 
made  the  other  way  were  the  right  ones.  But  if  we 
reject  the  idea  that  the  business  of  a  decision-making 
system  is  to  imitate  some  individual’s  decisions  (in 
which  case  the  only  point  of  building  the  system 
would  be  to  save  the  individual  the  trouble  of  making 
those  decisions  himself),  then  no  basis  remains  for 
asserting  that  the  decisions  made  by  one  procedure 
(e.g.,  by  the  commander)  are  inherently  appropriate 
simply  because  they  were  made  by  that  procedure, 
regardless  of  their  content.  An  examination  of  the 
merit  of  decisions  in  terms  of  their  content  is  a  matter 
of  intellectual  coherence  or  reliability,  not  validity. 

We  assert  also  that  intellectual  coherence  or  re¬ 
liability  is  very  measurable  and  is  in  fact  what  we 
want  the  output  of  a  decision-making  system  to  have. 
The  rest  of  this  philosophical  section  concerns  some 
thoughts  about  how  to  obtain  intellectual  coherence 
in  judgment-based  command  systems,  and  how  to 
establish  that  it  has  (or  has  not)  been  obtained. 

Before  going  on,  however,  we  pause  to  answer  a 
possible  objection.  It  is  possible  to  think  of  command 
situations  and  systems  within  which  the  quality  of 
system  performance  is  easily  defined  and  easily  mea¬ 
sured.  From  a  too-superficial  viewpoint  one  could 
argue,  for  example,  that  a  business  management 
should  maximize  dollar  return  and  that  dollar  return 
is  easily  measured.  Actually,  both  of  these  statements 
are  incorrect,  since  most  businesses  have  many  goals 
other  than  maximum  dollar  return  (e.g.  market  posi¬ 
tion,  stability  of  employment,  maintenance  of  stock 
value,  image,  etc.)  and  the  accounting  Fictions  under¬ 
lying  the  definition  of  profit  are  the  output  of  an 
elaborate  and  basically  subjective  judgmental  process. 
But  lower-level  command  systems  may  have  rather 
acceptable  external  standards  of  quality  of  per¬ 
formance.  The  goal  of  a  cab  dispatching  system,  for 
example,  might  be  to  minimize  total  customer  waiting 
time  — an  easily  measured  quantity. 

But  why  is  that  goal  appropriate,  rather  than  some 
other,  possibly  inconsistent  goal,  such  as  minimizing 
number  of  miles  traveled  by  empty  cabs?  We  can 
think  of  no  example  in  which  the  choice  of  goal,  or  of 
weighting  function  by  means  of  which  to  combine 
goals,  is  not  a  judgmental  matter.  So  even  command 
systems  in  which  performance  measures  are  easily 
identified  are  inherently  and  essentially  based  on 


value  judgments  — the  value  judgments  that  specify 
the  system’s  goals. 

A  task  analysis  of  command  systems 

As  we  envision  it,  the  basic  function  of  any  com¬ 
mand  system  is  to  make  decisions.  The  formal  analy¬ 
sis  of  decision-making  is  extensive  and  the  basic 
principles  are  well  understood.  We  will  not  review  that 
analysis  here;  such  references  as  Luce  and  Raiffa 
(1957)  present  it.  Implementing  that  analysis  in  the 
system  design  is,  we  believe,  necessary  to  attain  in¬ 
tellectual  coherence,  but  far  from  easy.  Table  10 
presents  a  detailed  breakdown  of  functional  steps  in 
implementing  the  analysis,  steps  that  must  be  per¬ 
formed  in  one  way  or  another  by  any  command 
system.  Table  10  also  contains  our  opinions  (very 
much  subject  to  modification)  about  whether  each 
function  should  be  performed  by  men,  machines,  or 
both,  and  our  opinions  (Firmer)  about  whether  each 
function  should  be  performed  at  the  moment  of  de¬ 
cision  or  in  advance. 

Functions  2  through  6  should  be  performed  ahead 
of  time  if  possible,  but  it  will  not  always  be  possible. 
We  are  particularly  interested  in  functions  5  and  10, 
which  along  with  function  2  (about  which  we  know 
nothing  abstract)  constitute  the  basic  functions  that 
men  should  perform  in  command  systems.  Functions 
5  and  10  correspond  to  the  two  basic  variables  of 
decision  theory:  utility  and  probability.  (Functions 
10  and  11,  as  stated,  imply  a  point  of  view  about 
how  relevant  probabilities  should  be  obtained  in  de¬ 
cision-making  systems;  systems  that  work  this  way 
are  called  Probabilistic  Information  Processing  or 
PIP  systems,  and  are  discussed  by  Edwards  and 
others  (see  Edwards,  1965;  Edwards,  Lindman,  and 
Phillips,  1964;  Edwards  and  Phillips,  1964;  Schum, 
Goldstein,  and  Southard,  1966;  Kaplan  and  New¬ 
man,  1966).  Both  of  these  quantities,  we  assert,  are 
inherently  judgmental;  the  basic  roles  of  men  in  com¬ 
mand  systems  are  to  make  these  judgments. 

Two  principles  for  judgment-based 

decision  system  design 

Table  10  implies  two  principles  for  the  design  of 
judgment-based  decision  systems.  These  principles 
have  somewhat  the  status  of  axioms; 

they  are  fundamental  to  our  argument,  but  not  di¬ 
rectly  demonstrable.  We  will  state  and  argue  for 
them,  but  firm  establishment  of  their  appropriate¬ 
ness  for  guidance  of  system  design  must  come  from 
success  of  the  resulting  systems. 

Principle  1 :  The  judgments  that  must  be  made  by 
men  in  decision  systems  should  be  fragmented  into 
relatively  small,  elementary  parts  when  possible. 
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Tabic  10- 


Functions  of  Command  Systems 


Function 

Performed 

By 

When  Performed 

1.  Recognize  that  a  decision  problem  exists 

men 

ahead  of  time 

2.  Identify  available  acts 

men 

ahead  of  time  if  possible 

3.  Identify  relevant  states  that  determine  payoff  for 
acts 

men 

ahead  of  time  if  possible 

4.  Identify  the  value  dimensions  to  be  aggregated 
into  the  payoff  matrix 

men 

ahead  of  time  if  possible 

5.  Judge  the  value  of  each  outcome  on  each  dimen¬ 
sion 

men 

ahead  of  time  if  possible 

6.  Aggregate  value  judgments  into  a  composite 
payoff  matrix 

machines 

ahead  of  time  if  possible 

7.  Identify  information  sources  relevant  to  discrimina¬ 

tion  among  states 

8.  Collect  data  from  information  sources 

men 

both 

ahead  of  time 
at  moment  of  decision 

9.  Filter  data,  put  into  standard  format,  and  display 
to  likelihood  estimator 

both 

at  moment  of  decision 

10.  Estimate  likelihood  ratios  (or  some  other  quan¬ 
tity  indicating  the  impact  of  the  datum  on  the 
hypotheses) 

men 

at  moment  of  decision 

11.  Aggregate  impact  estimates  into  posterior  dis¬ 
tributions 

machines 

at  moment  of  decision 

12.  Decide  among  acts  by  using  principle  of  maximiz¬ 
ing  expected  value 

machines 

at  moment  of  decision 

13.  Implement  the  decision 

both 

at  moment  of  decision 

We  see  five  advantages  to  using  this  principle. 

1.  It  permits  the  automation  of  significant  ele¬ 
ments  of  the  decision-making  task. 

2.  It  greatly  simplifies  the  task  of  the  human  beings 
working  in  the  system,  by  reducing  the  dif¬ 
ficulty  of  each  judgment. 

3.  Recause  of  2,  it  reduces  the  difficulty  of  train¬ 
ing  the  system  operators. 

4.  It  permits  allocation  of  the  judgment  task  to 
several  men  rather  than  one;  there  is  no  require¬ 
ment  that  all  information  must  ultimately  be 
evaluated  by  one  man.  In  systems  with  a  high 
information  load,  this  consideration  alone  would 
be  enough  to  justify  the  principle. 

5.  Because  of  1,  it  permits  machine  portions  of  the 
system  to  monitor  and  insure  intellectual  co¬ 
herences  of  various  kinds,  either  by  checking 
judgments  to  insure  coherence  or  by  perform¬ 
ing  operations  according  to  rules  that  guarantee 
coherence. 

Since  the  fifth  point  is  crucial,  it  is  appropriate  to 
add  that  examination  of  actual  human  decisions,  in 
laboratory  (see  Edwards,  1954)  and  other  contexts, 
indicates  that  incoherences  frequently  occur  in  them. 


In  fact,  the  known  rules  for  intellectual  coherence 
of  decisions  are  sufficiently  demanding  so  that  it  is 
extremely  difficult  by  unaided  intuition,  to  make  a 
reasonably  large  set  of  decisions  without  violating 
some  of  them.  So  principle  1,  properly  applied,  can 
be  expected  to  result  in  major  gains  in  coherence. 

Clearly  the  functional  analysis  of  Table  10  and  the 
use  of  principle  1  are  appropriate  only  if  men  can  in 
fact  perform  fragmented  judgments  effectively.  In 
an  important  sense,  it  is  self-evident  that  they  can, 
since  men  can  and  do  generally  make  reasonably 
appropriate  decisions,  and  some  version  of  each  of 
the  functions  listed  in  Table  5  must  be  performed, 
implicitly  or  explicitly,  before  any  decision  can  be 
made.  But  it  is  not  self-evident  that  men  can  perform 
such  functions  explicitly,  as  is  necessary  to  implement 
the  philosophy  of  system  design  implied  by  Table  10. 
Principle  2  asserts  that  they  can. 

Principle  2:  Men  can  make  explicit  probability 
and  value  judgments,  using  appropriate  response 
mechanisms  and  after  appropriate  training.  Where 
external  standards  of  correctness  of  such  judgments 
are  available,  human  judgments  will  usually  be  in- 
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correct,  but  not  severely  so,  and  appropriate  system 
design  can  minimize  such  errors.  Whether  or  not 
external  standards  of  correctness  are  available,  ap¬ 
propriately  obtained  judgments  will  be  relatively 
coherent. 

Principle  2  is  an  empirically  testable  assertion,  and 
a  variety  of  experimental  evidence  bears  on  it.  Psy¬ 
chophysics  is  the  branch  of  psychology  devoted  to 
the  extraction  of  human  judgments  about  reasonably 
simple  sensory  events;  a  basic  conclusion  is  that  men 
make  such  judgments  very  well  indeed.  (See  Stevens 
and  Galanter,  1957.)  Recent  research  on  probability 
estimation  indicates  that  men  can  judge  relative 
frequencies  with  great  accuracy  (Robinson,  1964; 
Shuford,  1961).  More  complex  probability  judg¬ 
ments  suffer  from  an  inherent  deficiency  that  has  been 
named  conservatism;  men  are  unable  to  extract  from 
data  as  much  certainty  as  the  data  justify.  (See  Peter¬ 
son  and  Miller,  1965;  Phillips,  Hays,  and  Edwards, 
1966.)  But  human  conservatism  in  probability  estima¬ 
tion  can  be  overcome  by  appropriate  system  design; 
that  is  the  point  of  the  PIP  system  mentioned  above. 
(See  Edwards,  Lindman,and  Phillips,  1964.)  Evidence 
on  human  value  judgments  is  sketchy  and  frag¬ 
mentary;  such  as  it  is,  it  indicates  that  men  do  rather 
well  at  translating  even  rather  complicated  value 
systems  into  numbers.  (See,  for  example,  Yntema 
and  Torgerson,  1961.) 

But  final  justification  of  the  use  of  Principles  1  and 
2  for  system  design  can  come  only  from  success 
of  the  resulting  systems. 

Judgmental  systems  are  self-validating 

The  arguments  presented  above  have  led  us  to  a 
set  of  ideas  about  how  to  design  judgmental  systems. 
One  crucial  feature  of  these  ideas  is  that  explicit 
value  or  utility  judgments  lie  at  the  core  of  such 
systems.  This  fact  has  implications  for  the  validation 
(in  the  sense  already  defined)  of  such  systems. 

As  we  have  pointed  out,  validation  in  a  classical 
sense  consists  of  demonstrating  coherence  between 
the  output  of  a  system,  in  this  case  decisions,  and  the 
comparable  output  of  some  other  system  considered 
to  be  effective  or  valid.  With  decision  systems,  how¬ 
ever,  such  acceptable  external  criteria  are  nonexistent. 
The  usual  expedient  is  to  use  some  wise  and  experi¬ 
enced  decision-maker's  judgments  as  the  criterion. 
However,  unless  the  goal  of  the  system  is  merely  to 
reproduce  that  man’s  decisions,  this  procedure  is  un¬ 
satisfactory —especially  since  any  man’s  decisions 
are  likely  to  be  incoherent  to  some  extent. 


As  we  have  already  argued,  intellectual  coherence, 
taken  over  as  large  a  domain  of  thought  as  possible, 
is  an  alternative  approach  to  validation,  and  the  only 
approach  available  for  the  command  systems  of  in¬ 
terest  here.  And  a  variety  of  kinds  of  coherence  are 
built  into  the  sort  of  judgmental  system  implied  by 
Table  10. 

Still  other  kinds  of  coherence  can  and  should  be 
examined  by  means  of  research  on  any  proposed  com¬ 
mand  system;  such  research  can  range  from  highly 
informal  studies  of  system  elements  to  major  formal 
simulations  of  the  system  as  a  whole. 

In  the  particular  case  of  decision  systems  based  on 
value  judgments,  a  natural  requirement  of  the  system 
is  that  the  decisions  should  cause  as  much  value  as 
possible  to  accrue  to  the  entity  in  whose  service  the 
decisions  are  being  made.  Procedures  internal  to 
the  system  will  guarantee  this,  given  that  the  value 
judgments  made  by  system  operators  are  taken  as 
the  “true”  values.  The  question  of  whether  this 
criterion  is  met  therefore  reduces  to  two  other  ques¬ 
tions.  Are  value  judgments  reliable,  from  time  to 
time  within  one  judge  or  from  one  judge  to  another? 
If  not,  can  the  unreliability  be  accounted  for  as  true 
differences  in  values,  from  time  to  time  or  from  one 
judge  to  another?  This  second  question  is  bound  to  be 
a  matter  of  opinion,  since  “true”  values  are  inac¬ 
cessible  and  perhaps  undefined.  Still ,  the  opinion  need 
not  be  entirely  unguided  by  data. 

Obviously  such  questions  of  reliability  will  be  ex¬ 
tensively  studied  in  the  course  of  system  design.  So 
will  other  forms  of  intellectual  coherence.  By  the  time 
any  command  system  of  the  kind  implied  by  Table  10 
has  been  fully  designed,  formal  and  empirical  informa¬ 
tion  about  the  various  relevant  kinds  of  intellectual 
coherence  will  have  been  built  into  the  design  details 
at  almost  every  point.  Thus  the  design  of  such  a  sys¬ 
tem  is  self-validating,  in  a  very  important  sense. 

Of  course  no  validation  is  ultimate;  each  conclusion 
that  a  system  is  valid  for  its  purpose  means  no  more 
than  that  a  decision  has  been  reached  to  proceed  to 
the  next  step  in  its  design  and  use.  Conclusions 
reached  during  system  design  concerning  the  level 
of  quality  to  be  expected  in  system  performance  will 
be  modified  or  replaced  by  conclusions  based  on  the 
result  of  simulations;  conclusions  based  on  simula¬ 
tions  will  be  modified  or  replaced  by  conclusions 
based  on  actual  system  use.  And  even  actual  use  is 
not  an  ultimate  criterion;  each  new  use  under  new  con¬ 
ditions  may  require  a  new  judgment  of  system  validity. 
Because,  in  the  last  analysis,  all  we  can  ever  mean 
by  stating  a  procedure  is  valid  is  that,  on  the  basis  of 
what  we  know,  it  makes  sense. 
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New  directions  for  information  systems  through  advances 
in  machine  organization 


by  James  P.  Anderson 
Fort  Washington,  Pennsylvania 


INTRODUCTION 

While  automated  information  systems  design  has 
made  considerable  progress  over  the  past  Five  or 
six  years,  it  seems  apparent  that  it  has  fallen  far 
short  of  the  promises  and  the  concepts  advanced 
during  the  era  of  the  large  scale  command  and  con¬ 
trol  systems  that  fostered  its  development.  Part  of 
the  failure  to  realize  the  lofty  ambitions  of  that 
time  appear  to  be  related  to  our  concepts  of  computers 
and  the  ways  we  are  forced  to  use  them. 

Presently,  we  have  a  new  opportunity  to  affect  the 
structure  of  a  new  generation  of  machines.  This 
opportunity  will  be  occasioned  principally  because 
of  the  very  rapid  reduction  in  logic  costs  being 
brought  about  by  the  developments  in  microcircuit 
technology. 

It  is  not  the  intent  of  this  paper  to  offer  pat  solutions 
to  the  problems  we  all  agree  are  quite  difficult;  rather, 
it  is  to  suggest  several  approaches  that  are  feasible  in 
the  context  of  the  new  hardware  technology,  and  from 
this  perhaps  stimulate  contributions  to  machine  or¬ 
ganization  from  information  systems  designers.  In 
this  way,  we  may  arrive  at  machines  that  will  assist 
them  in  solving  basic  problems  rather  than  overcoming 
the  inadequacies  of  the  tools  that  are  given  to  work 
with. 

Direct  execution  machines 

One  of  the  continuing  and  pervading  problems  in  the 
design  and  development  of  information  systems  has 
been  what  has  come  to  be  known  as  the  programming 
problem.  While  not  all  of  the  “problem”  can  be 
attributed  to  the  nature  of  the  computer  systems  we 
work  with,  sufficient  difficulty  has  been  encountered 
to  force  the  adoption  of  one  of  the  various  higher 
level  programming  languages  to  increase  the  intelligi¬ 
bility  and  decrease  the  time  required  for  preparation 
of  programs. 

Even  with  the  present  programming  languages, 
much  time  is  spent  in  compiling  and  debugging,  the 
latter  frequently  involving  peculiar  quirks  of  a  par¬ 


ticular  machine,  for  example,  how  it  encodes  data. 
From  the  point  of  view  of  resources  used,  one  must 
also  consider  the  large  amount  of  effort  expended  by 
the  manufacturers  and  various  systems  development 
groups  in  preparing  compilers,  diagnostic  programs, 
utilities  and  the  like  to  create  an  environment  for 
programming  information  systems.  The  costs  which 
are  ultimately  passed  on  to  the  users  become  almost 
incalculable. 

In  addition,  in  order  to  permit  user  independence 
from  particular  hardware  there  has  been  a  major  effort 
at  standardizing  the  various  languages  as  well  as 
adoption  by  the  various  services  and  government 
agencies  of  one  of  the  languages  as  a  required  standard 
for  their  problems.  The  Air  Force,  for  example,  has 
adopted  JOVIAL  as  the  standard  for  command  and 
control  systems  while  the  DOD  has  placed  increased 
emphasis  on  use  of  COBOL  for  many  applications. 

As  a  result  of  these  language  standardization  efforts, 
one  might  consider  establishing  requirements  for 
machines  that  directly  execute  programs  in  one  of  the 
standard  languages  as  a  means  of  simplifying  some  of 
the  programming  problems.  Earlier  work  has  shown 
the  feasibility  of  this  approach  (Anderson,  1961; 
Bashkow,  1966).  With  current  software  costs  exceed¬ 
ing  the  development  costs  of  hardware,  the  economic 
pressure  for  adoption  of  this  kind  of  approach  will 
become  even  greater  for  the  next  round  of  machines. 

With  the  objection  to  direct  execution  machines 
based  on  the  additional  cost  of  logic  being  rapidly 
eroded,  the  only  remaining  objection  centers  mainly 
on  the  single  language  capability  of  such  a  machine. 
This  objection  would  not  be  relevant  for  those  situa¬ 
tions  w  here  de  facto  standards  are  insisted  upon  for 
various  reasons,  and  could  be  overcome  altogether 
by  structuring  such  machines  to  have  the  language 
analysis  logic  separable  (perhaps  by  plug-in  units) 
from  the  main  part  of  the  machine.  The  current  emula¬ 
tor  technology  suggests  that  a  basic  machine  structure 
can  be  adapted  to  a  variety  of  purposes  by  varying 
the  logic  itself. 
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It  is  not  suggested  that  the  direct  execution  ma¬ 
chines  will  be  applicable  in  all  information  system 
situations,  but  for  many  applications,  particularly 
those  involving  dedicated  systems,  this  approach 
could  be  quite  powerful.  When  combined  with  on-line 
techniques,  one  might  even  anticipate  simpler  control 
programs  by  reducing  the  interface  with  compilers, 
and  providing  direct  language  debugging. 

Parallel  organizations 

The  present  generation  of  computer  systems  has 
been  different  from  its  predecessors  in  one  major 
respect,  that  is,  nearly  every  manufacturer  offers  a 
multiprocessor  version  of  one  or  more  machines  in 
his  line.  The  initial  emphasis  on  this  kind  of  organiza¬ 
tion  and  the  various  multicomputer  organizations  that 
were  both  contemporary  and  preceded  it,  were  for 
reliability  and  systems  balance.  However,  due  to  the 
R&D  on  executive  systems  for  multiprogramming 
control  for  both  time-sharing  applications  and  multi¬ 
processor  management,  it  became  apparent  that  it 
would  be  possible  to  exploit  the  multi-processor 
structure  to  give  better  performance  through  parallel 
operation  on  a  single  problem.  To  this  end,  some  of 
the  control  languages  for  present  machines  have  been 
enriched  to  permit  programmer  specification  of 
opportunities  for  parallel  execution  of  segments  of 
the  same  program.  While  this  is  an  important  step, 
it  is  insufficient  to  achieve  the  desired  levels  of  per¬ 
formance  in  information  systems. 

Under  different  assumptions  from  those  in  the 
preceding  section  of  this  paper,  it  is  clear  that  it 
would  be  possible  to  achieve  quite  spectacular  im¬ 
provements  in  performance  by  systematically  ex¬ 
ploiting  all  parallelism  inherent  in  every  program. 

In  order  to  be  most  effective,  such  systematic  ex¬ 
ploitation  must  be  based  on  automatic  detection  of 
opportunities  for  parallel  execution.  Reliance  on 
programmer  specification  of  such  opportunities  is 
both  unnecessary  and  burdensome. 

The  programming  technology  is  sufficiently  ad¬ 
vanced  to  handle  parallelisms  in  the  fine.  A  method 
for  detecting  parallelism  in  algebraic  languages  is 
outlined  in  the  appendix.  Another  method,  and  a  sys¬ 
tem  structure  for  exploiting  parallelisms  in  algebraic 
expressions  is  outlined  in  an  article  in  the  February 
1966  issue  of  the  IEEE  Transactions  on  Electronic 
Computers  (Hellerman,  1966). 

While  exploiting  parallelism  in  computational 
languages  is  certainly  feasible,  one  area  that  would 
be  more  fruitful  to  examine  for  information  systems 
application  is  the  possibility  of  organizing  highly 
parallel  systems  for  data  processing.  The  basic 
techniques  are  apparent:  Data  partitioning,  exploiting 


repetitive  operations  on  files  (such  as  extraction  of  a 
subset  of  a  file  for  subsequent  processing)  and  exploit¬ 
ing  parallelism  at  the  statement  level  in  a  manner 
similar  to  that  found  in  the  proposals  for  purely 
algebraic  languages. 

In  addition  to  multiple  processor  structures  already 
in  existence,  the  design  of  highly  parallel  data  pro¬ 
cessing  oriented  systems  requires  many  more  channels 
into  and  out  of  disc  files  or  other  mass  storage  devices 
than  are  presently  available.  It  is  possible  that  virtual 
channels  patterned  after  the  multiplexed  drum 
technique  used  on  the  GE  645  will  provide  a  nec¬ 
essary  increase  in  performance  when  applied  to  head 
per  track  disc  files. 

The  reason  highly  parallel  data  processing  systems 
are  believed  to  be  important  is  the  improvement  in 
overall  performance  that  would  accrue  to  almost  any 
kind  of  information  system  through  the  reduction 
in  sorting  time  alone,  using  data  partitioning  tech¬ 
niques. 

The  present  lack  of  a  generalized  multi-file  data 
retrieval  and  reporting  program  on  any  systems  known 
to  the  author  seems  to  be  related  at  least  in  part  to 
the  sorting  that  would  be  required. 

With  very  few  exceptions  there  has  been  little  in¬ 
novation  to  the  basic  structure  of  general  computing 
systems  since  Mauchly  and  Eckert  designed  EDVAC 
and  UNIVAC.  The  reasons  for  using  highly  stylized 
encoded  instructions  for  commands  are  being  rapidly 
dissipated,  as  are  the  reasons  for  the  treatment  of 
programs  as  sequential  processes.  The  many  problems 
in  information  systems  that  are  tied  to  programming 
and  performance  may  be  unnecessary  and  exist  only 
because  we  choose  to  live  with  information  systems 
that  are  not  suited  to  our  requirements.  The  means 
for  changing  this  situation  is  at  hand,  and  it  seems 
fairly  certain  that  new  machines,  both  direct  execution 
and  highly  parallel,  will  be  developed. 

New  developments  will  not  be  limited  to  the  two 
suggested  above,  nor  is  it  intended  to  suggest  that 
direct  execution  machines  and  highly  parallel  organi¬ 
zations  will  solve  all  problems  in  automated  informa¬ 
tion  systems.  What  is  intended  is  to  suggest  that  in¬ 
sufficient  attention  is  paid  to  the  design  of  proper 
hardware  tools  for  this  kind  of  problem,  compound¬ 
ing  the  software  problems  that  are  enormous  enough 
in  themselves. 

Appendix  A:  an  approach  to  explicating  parallelism 

in  algebraic  languages 

Nature  of  sequentiality  and  parallelism 

The  study  of  parallelism  in  programs  is  in  fact  a 
study  of  required  sequentiality  within  a  program. 
There  are  two  sources  of  sequentiality  (and  parallel- 
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ism)  arising  from  the  precedence  of  the  arithmetic 
operator,  and  the  sequentiality  enforced  by  the  avail¬ 
ability  of  the  operands. 

As  an  example  of  the  first  form,  consider  the  expres¬ 
sion  a  b-c/d-he.  Given  the  machine  capability  one 
could  compute  the  partial  results  a-b  and  c/d  in  par¬ 
allel  before  proceeding  with  the  sequential  operations 
of  subtracting  the  partial  results  and  adding  e.  A  triv¬ 
ial  example  of  the  second  form  is  shown  below: 

(1)  A  =  B 

(2)  C  =  D 

(3)  E=A  +  B 

Here  the  first  two  assignment  statements  could  be 
executed  in  parallel,  while  the  third  would  have  to 
be  deferred  until  the  first  statement  was  completed. 

The  simple  examples  shown  above  illustrate  two 
of  the  basic  concepts  involved  in  the  detection  of 
implicit  parallelism  within  programs.  There  is  one 
additional  concept  of  importance,  that  of  “regions.” 
The  region  is  the  span  over  which  implied  parallel¬ 
isms  can  be  logically  sought.  The  nature  of  regions  is 
closely  related  to  the  notion  of  scope  found  in  the 
constructs  of  ALGOL.  Examples  of  regions  are  the 
scope  of  a  For  Statement,  the  true  clause  of  a  Con¬ 
ditional  Statement,  the  false  clause  of  the  same  kind 
of  statement,  Compound  Statements,  etc.  In  general, 
the  regions  are  bounded  by  statements  that  interrupt 
the  implied  control  flow. 

In  addition  to  the  natural  parallelism  of  programs, 
there  are  a  number  of  special  cases  that  occur  with 
sufficient  frequency  to  warrant  inclusion  in  any  con¬ 
sideration  of  automatic  analysis  of  implied  parallelism. 
One  construct  that  can  be  exploited  is  repetitive 
operations  on  vectors  and  arrays.  Thus,  the  inner 
product  of  two  vectors  could  be  recoded  as  a  set  of 
individual  multiplications  and  a  tree  of  additions, 
or  as  a  set  of  interactions  over  segments  of  the  vectors. 
Similarly,  the  multiplication  of  two  conformable  ma¬ 
trices  can  be  expanded  into  independent  computation 
of  each  element  of  the  resultant  product  matrix.  These 
special  cases  will  not  be  discussed  further  here. 

Techniques  for  analyzing  intra-expression  and 

inter-statement  parallelism 

The  basic  technique  for  analyzing  and  detecting 
intra-expression  parallelism  is  to  convert  the  expres¬ 
sion  into  a  tree  form  using  techniques  of  algebraic 
expression  analysis  found  in  compilers.  The  execution 
order  for  each  binary  operator  in  the  expression  can 
be  derived  directly  by  associating  with  each  operand 
and  partial  result  entering  into  the  compilation,  a 
level  number  derived  according  to  the  following  rules: 

(1)  All  variables  (including  constants)  are  of  level  0. 


(2)  For  each  partial  compilation,  find: 

L,.  =  MAXfL^LjFH,  where  Lr  is  the  level  of 
the  result 

Lt  is  the  level  of  operand  1 
L2  is  the  level  of  operand  2 
The  level  numbers  are  the  execution  order  for  each 
of  the  operators  in  the  expression.  As  an  example,  the 
expression 

(b+c)/(e— f>bd— (r+p)/(q— m)-hr 
would  result  in  the  following  tree: 
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The  numbers  indicate  the  order  in  which  the  nodes 
are  generated  by  the  scan.  The  scan  would  be  able 
to  generate  pseudo  three  address  code  with  the  ex- 
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Thus,  the  expression  would  permit  the  operations 
on  lines  1,  2,  5,  and  6  to  be  executed  in  parallel  first. 
Following  this,  the  operations  on  lines  3  and  7  could 
be  executed  in  parallel,  after  which  the  operations  on 
lines  4,  8  and  9  would  have  to  be  executed  in 
sequence. 

The  technique  described  above  depends  only  on 
the  precedence  (hierarchy)  of  the  operators  in  the 
language  being  analyzed. 

The  treatment  of  inter-statement  parallelism  de¬ 
pends  on  recognition  of  the  regions,  and  an  analysis 
of  the  availability  of  variables.  This  is  merely  another 
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way  of  saying  that  one  cannot  logically  use  the  result 
of  a  computation  before  it  is  available.  As  an  ex¬ 
ample,  consider  the  following  statements: 

(1)  A  =  B  +  C 

(2)  D  =  F  +  1 

(3)  E  =  B  +  A/D 

Statements  1  and  2  could  be  executed  in  parallel. 
However,  statement  3  would  have  to  be  deferred  until 
the  results  of  the  previous  operations  were  available. 
In  this  case,  the  results  of  the  previous  operations 
are  available  simultaneously  with  a  level  value  of  1 
which  would  permit  initiation  of  the  third  statement 
at  the  very  next  instant.  If,  however,  the  first  state¬ 
ment  were  the  result  of  the  expression  used  to  illus¬ 
trate  intra-expression,  the  third  statement  of  the  region 
would  have  to  be  deferred  until  its  completion. 

By  noting  that  the  level  value  of  the  final  result 
defines  the  earliest  available  time  for  that  variable, 
we  are  able  to  extend  the  techniques  used  for  intra¬ 


expression  analysis  to  cover  the  analysis  of  regions 
by  affixing  in  the  symbol  table  the  level  number 
achieved  by  each  variable  as  assignments  are  made  to 
it.  Now,  rather  than  using  zero  for  the  level  value  of 
variables,  we  can  use  the  level  value  stored  with  the 
variable  in  the  symbol  table,  and  apply  the  algorithm 
noted  above. 

The  techniques  noted  above  can  be  applied  during 
compilation  to  explicate  all  of  the  natural  implied  par¬ 
allelism  within  a  program. 

1  J.  P  ANDERSON  A  computer  for  direct  execution  of 

algorithmic  languages  In  AFIPS  Conference  Proceed¬ 
ings:  1961  Eastern  Joint  Computer  Conference.  184- 
193  1961 

2  T.  R.  BASHKOW,  A.  SASSON  and  A.  KRONSFFLD 
System  Design  of  a  FORTRAN  Machine .  July  1966 

3  H.  HELLFRMAN  Parallel  processing  of  algebraic  ex¬ 

pressions.  IEEE  Transactions  on  Electronic  Computers 
Vol.  EC- 15  No.  1  82-91  February  1966 
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INTRODUCTION 

Policy  statements  are  for  the  most  part  expressed  in 
general  terms.  This  generality  provides  the  broad  cov¬ 
erage  usually  desired  of  such  statements  and  at  the 
same  time  allows  for  flexibility  in  application.  The 
generality  of  policy  statements,  moreover,  serves  still 
another  purpose.  Such  statements  tend  to  be  polit¬ 
ical  in  nature  where  political  can  be  taken  to  mean 
“in  the  context  of  a  people  and  organizational  en¬ 
vironment  of  diverse  and  at  times  conflicting  back¬ 
grounds,  purposes,  and  goals.”  In  such  a  context 
useful  statements  must  be  of  such  generality  and,  at 
times,  ambiguity  to  allow  for  agreement  among  the 
differing  people  and  organizational  elements.  Many 
of  the  current  concepts  being  applied  to  command 
control  systems  are  essentially  policy;  that  is,  they 
exhibit  a  political  character  including  generality  and 
ambiguity.  Application  of  these  policies  or  concepts 
to  command  and  control  systems  requires  specific 
interpretation. 

In  essence,  command  control  systems  are  defined 
by  policies  and  concepts  appropriate  to  a  political  en¬ 
vironment.  In  applying  these  policies  and  concepts 
to  the  technical  development  of  command  control  sys¬ 
tems,  the  generality  and  ambiguity  inherent  in  our  def¬ 
initions  create  problems.  For  purposes  of  develop¬ 
ment,  we  need  definitions  which  are  scientific  rather 
than  political  in  nature. 

This  paper  explores  some  of  the  problem  areas 
which  are  brought  about  by  the  application  of  essen¬ 
tially  politically-defined  concepts  to  a  decision-making 
and  developmental  effort  which  is  essentially  technical 
in  nature. 

Command  control  systems 

There  is  no  established  exact  definition  of  “com¬ 
mand  control  systems.”  The  term  seems  to  be  one  that 
when  used  means  what  one  wants  it  to  mean,  to  para¬ 
phrase  Humpty  Dumpty.  An  example  of  this  is  the 
high  probability  of  finding  in  most  papers  that  discuss 
command  control  systems  a  phrase  such  as  “For  pur¬ 
poses  of  this  article,  command  control  systems  are 
defined  as. . . .” 


Nominal  definitions 

One  document  (Parsons  and  Perry,  1965)  cites 
some  10  different  nominal  definitions  of  command 
control  systems  (pp.  73-77)  in  addition  to  their  own 
notion  that  “a  descriptive  analysis  constitutes  the 
‘definition’”  (p.  7).  Most  of  the  nominal  definitions 
cited  have  in  common  an  assertion  such  as  “a  com¬ 
mand  and  control  system  consists  of  all  the  equip¬ 
ments,  people,  procedures,  etc.,  needed  to  deter¬ 
mine  and  direct  courses  of  action  for  assigned 
forces.”  Such  definition  may  be  useful  as  an  encom¬ 
passing  term  for  general  discussion.  At  an  appli¬ 
cation  level  it  is  too  encompassing  and  that  is  its 
weakness.  It  encompasses  systems  of  decidedly  differ¬ 
ent  characteristics.  It  includes  not  only  those  sys¬ 
tems  at  superordinate  levels  primarily  for  planning 
and  management  actions  but  also  those  systems  at 
subordinate  levels  for  immediate  direction  of  forces, 
e.g.,  aircraft  control. 

Categorization 

Attempts  to  reduce  this  very  general  definition  by 
categorizing  (or  classifying)  have  been  made.  For 
example,  “strategic  versus  tactical,”  or  “command 
versus  control”  or  (in  another  vein)  “real  time 
versus  nonreal  time”  have  all  been  used.  These  clas¬ 
sification  schemes  give  the  appearance  of  narrowing 
the  definition  since  they  seem  to  subdivide  the  sys¬ 
tems  into  smaller  groupings.  This  help  is,  for  the  most 
part,  illusory  as  the  problem  is  right  where  it  was 
before;  namely,  using  often  ill-defined  words  to  define 
an  ill-defined  concept. 

In  his  paper  for  the  Second  Congress,  Benington 
distinguished  between  strategic  and  tactical  command 
control  systems,  where  “strategic”  and  “tactical” 
were  used  in  their  classical  sense  (Benington,  1965). 
That  paper  carried  the  distinction  one  step  further. 
Benington  listed  (at  least  by  example)  those  systems 
he  wished  to  be  considered  as  “strategic.”  Thus  was 
added  to  his  definition  the  most  important  factor  of 
“pointing  to”  or  providing  “observability”  to  the 
definition. 
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In  most  cases  (the  above  is  cited  as  an  example)  the 
technique  has  been  employed  on  an  after-the-fact 
basis.  In  other  words,  existing  systems  are  listed  as 
belonging  in  one  category  or  another  without  nec¬ 
essarily  explicitly  establishing  the  criteria  of  classi¬ 
fication.  The  utility  of  this  approach,  if  used  to  classi¬ 
fy  a  new  or  proposed  system,  depends  heavily  on 
familiarity  with  the  systems  as  categorized  for  com¬ 
parison.  Nevertheless  the  notion  of  “observability” 
is  important  in  clarifying  the  definition  of  command 
and  control  systems. 

Explicit  criteria  for  classification 

As  implied  above,  the  addition  of  explicit  criteria 
to  an  observable  classification  scheme  would  reduce 
ambiguity  of  definition  and  thus  improve  communi¬ 
cation.  This  in  turn  suggests  that  criteria  them¬ 
selves  be  observable  characteristics;  examples  of 
this  approach  follow. 

Sensing  — first,  one  may  consider  the  means  by 
which  the  systems  sense  the  environment  whether 
largely  automatic,  e.g.,  radar,  or  largely  manual, 
e.g.,  messages  prepared  by  people.  Such  a  dimension 
seems  readily  reducible  to  quantification,  e.g.,  the 
percentage  of  inputs  of  each  kind. 

Effect  — a  similar  dimension  has  been  previously 
suggested  as  . .  the  degree  to  which  the  system  is 
automatically  coupled  to  the  environment  that  it 
must  sense  and  effect”  (Shaw,  1966). 

The  latter  distinction  in  fact  provides  two  dimen¬ 
sions,  the  sensing  factor  discussed  and  the  effect  or 
action  dimension.  This  latter  dimension  relates  to 
the  degree  of  interposition  of  people  between  the  com¬ 
puter  output  and  the  action  point.  Again,  quantifi¬ 
cation  in  terms  of  percentages  might  be  useful. 

System  response  time  — as  was  noted  earlier,  the 
“real  time  versus  nonreal  time”  classification  has 
at  times  been  used.  In  general,  however,  we  do  not 
have  adequate  definition  of  real  time  as  applied  to 
command  control  systems.  One  definition  of  real  time 
is  that  in  any  simulation  in  which  the  time  dimension 
was  modeled  on  a  ratio  of  1 : 1  to  real  time  the  system 
was  tautologically  called  “real  time”  (Gill,  1966). 
This  definition  rules  out  most  command  control  sys¬ 
tems  as  real  time.  Generally,  however,  when  we  speak 
of  command  control  systems  as  real  time  we  mean  a 
system  which  must  perform  some  function  (or  calcu¬ 
lation)  with  sufficient  speed  to  ensure  satisfactory 
performance.  A  good  example  is  intercept  direction 
via  data  link  in  which  the  vectoring  calculations  are 
done  in  a  time  scale  faster  than  real  time;  that  is,  the 
relative  aircraft  positions  are  projected;  i.e.,  pre¬ 
dicted  in  less  time  than  the  aircraft  can  actually  fly  the 
projected  distance.  In  this  sense,  then,  we  mean  more 


precisely  response  time  rather  than  real  time.  Real 
time  in  this  sense  connotes  a  time  restriction  imposed 
on  the  system  by  some  environmental  condition. 

Unfortunately,  the  term  “response  time”  is  often 
taken  to  mean  the  elapsed  time  between  request  by  a 
user  and  receipt  of  an  output  (display)  from  the  com¬ 
puter.  Usage  of  the  term  “response  time”  in  this  sense 
seems  to  be  quite  common  in  discussions  of  time¬ 
sharing  systems. 

Since  the  term  is  quite  applicable  in  both  cases,  it 
is  proposed  that  the  former  be  called  “system  re¬ 
sponse  time-  and  the  latter,  “display  response  time.” 

There  are  at  least  four  possible  ways  to  quantify  on 
the  dimension  of  system  response  time:  (1)  The  “tight¬ 
ness”  of  the  response  time  requirements;  (2)  the 
number  of  functions  having  a  response  time  require¬ 
ment;  (3)  the  portion  of  computer  time  devoted  to 
meeting  response  time  requirements,  and  (4)  the 
predictability  of  the  response  time  requirement. 

User  interaction  — another  dimension  to  be  called 
“user  on-line  versus  off-line”  would  deal  with  the 
means  by  which  users  interacted  with  the  computer 
and  program.  Consideration  and  quantification  of 
display  response  time,  user  input  time,  ratio  of  people 
intervention  and  nonintervention  to  user  inputs  and 
outputs  would  be  the  means  of  measuring  such  a 
dimension. 

Further  consideration  for  definition 

It  is  obvious  that  the  kinds  of  criteria  discussed 
above  may  be  applied  equally  well  to  any  automated 
information  system.  There  is  nothing  about  them 
which  inherently  enables  one  to  distinguish  a  com¬ 
mand  control  system  from  any  other  system.  Of 
course,  if  the  work  of  obtaining  measurements  were 
done  with  care,  one  might  be  able  to  distinguish 
among  command  control  systems. 

It  must,  therefore,  be  the  case  that,  if  system 
characteristics  in  the  physical  sense  do  not  separate 
one  kind  of  system  from  another,  something  tran¬ 
scends  the  usual  physical  characteristics  which  does 
enable  such  a  distinction.  It  would  seem  that  the 
intent  of  the  system  in  this  instance  is  the  distin¬ 
guishing  factor;  that  is,  a  common  control  system  is 
intended  to  provide  command  control.  Since,  as  seems 
to  be  the  case,  it  is  relatively  easy  to  define  the 
physical  dimensions;  that  is,  the  system  part  of  the 
term,  then  the  difficulty  must  be  with  the  “command 
control”  part  of  the  term.  It  would  be  consistent 
with  the  basic  notion  of  this  paper  if  “command  con¬ 
trol”  were  the  political  part  and  “system”  the  techni¬ 
cal  part  of  the  phrase  “command  control  system.” 

Current  situation  summarized 

Current  definitions  of  command  control  systems 
seem  to  rest  on  the  assumption  of  inherent  difference 
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between  command  control  systems  and  other  systems. 
For  that  class  of  systems  to  which  the  label  “command 
control”  is  applied,  no  definitions  useful  to  dis¬ 
tinguishing  at  the  level  of  application  exist.  In  this 
sense,  application  includes  interpretation  of  policy 
to  a  particular  case.  Little,  if  any,  work  has  been  done 
to  enable  distinction  among  command  control  systems 
by  measurable  criteria. 

Current  concepts 

Continuing  with  the  theme  that  many  problems  are 
brought  about  by  attempting  to  apply  essentially 
political  definitions  to  technical  decisions  and/or 
development,  three  important  concepts  are  examined. 

Standardization 

A  most  important  trend  today  in  the  development  of 
automated  command  control  systems  is  the  emphasis 
on  standardization.  This  has  some  varying  inter¬ 
pretations  but  in  general  seems  to  mean  standardized 
computer  equipment  packages  as  well  as  standardiza¬ 
tion  on  what  some  people  have  called  “nonfunctional 
software.”  The  term  “nonfunctional  software”  gen¬ 
erally  seems  to  refer  to  such  things  as  the  compiler 
systems  (implying  a  standard  language),  the  utility  or 
program  production  systems,  certain  kinds  of  gen¬ 
eralized  data  management  capabilities,  and  in  some 
cases  includes  an  executive  program  capability  of  a 
type  which  is  found  most  often  in  operating  systems. 

Incorporation  of  data  management  capabilities 
seems  to  be  consistent  with  the  major  conclusion  of 
a  review  of  seven  command  control  systems  which 
was:  “The  major  trend  in  computer  programming  for 
military  command  and  control  systems  seems  to  be 
the  trend  toward  the  development  of  generalized, 
user-oriented  information  systems”  (Shaw,  1966). 

These  are  often  referred  to  as  data  management  sys¬ 
tems  wherein  a  generalized  capability  to  structure, 
enter,  update,  retrieve,  and  format  outputs  on  some 
set  of  information  files  is  provided  by  means  of  a 
user-oriented  language  which  enables  the  user  to 
manipulate  the  data  at  his  disposal.  Essentially,  a 
language  for  nonprogrammers  is  created  which  allows 
them  to  do  things  which  in  other  systems  only  pro¬ 
grammers  could  do. 

Inclusion  of  data  management  and  an  executive 
capability  begins  to  stretch,  however,  the  adequacy 
of  the  term  “nonfunctional.”  Certainly  the  “user- 
oriented  information  system”  is  intended  to  be  quite 
functional.  It  would  seem  that,  whether  intentional 
or  not,  we  have  a  definition  which,  while  perhaps 
adequate  in  a  political  sense,  is  inadequate  in  con¬ 
veying  the  technical  connotations. 

In  distinction  to  the  definition  of  standardization 
as  given  above  is  the  realization  that  it  is  no  longer 


appropriate  to  consider  the  computer  or  computer 
program  as  just  an  isolated  processing  tool.  Rather, 
we  have  grown  in  our  understanding  to  the  extent 
that  we  tend  to  consider  the  computer  and  programs 
as  one  element  or  subsystem  of  a  larger  system 
doing  information  processing.  Among  these  other 
elements  or  subsystems  within  the  broader  system 
context  are,  for  example,  the  man-machine  interface, 
the  communication  system,  and  the  information  sub¬ 
system.  More  recently,  there  seems  to  be  a  slowly 
growing  recognition  that  any  one  given  system 
(within  a  major  command,  for  example)  deals  only 
with  a  portion  of  the  total  information  base  with 
which  the  command  must  deal. 

Standardization  as  defined  tends  to  be  in  conflict 
with  the  growing  recognition  of  the  computer  and 
programs  as  one  subsystem  of  several  making  up  the 
automated  information  system.  If  one  accepts  the 
latter  view,  then  it  seems  quite  clear  that  some, 
perhaps  substantial,  standardization  of  the  other 
major  subsystems  is  necessary  to  solve  the  problem 
within  the  computer  and  program  area. 

The  degree  of  standardization  of  the  other  sub¬ 
systems  which  one  thinks  is  required  may  well  depend 
on  the  kind  of  command  control  system  one  is  talking 
about.  For  example,  consider  the  two  dimensions  of 
automatic  versus  manual  sensing  and  user  on-line 
versus  off-line  discussed  above  (Explicit  criteria 
for  classification). 

For  a  system  with  high  manual  and  low  automatic 
sensing  only  minimal  standardization  in  the  sensing 
and  communications  would  be  required.  On  the  other 
hand,  if  the  same  system  had  a  big  user  on-line 
loading,  then  substantial  standardization  to  the 
display  subsystem  would  be  required. 

Whether  one  views  the  need  for  standardization 
(to  some  degree)  of  the  other  subsystems  in  order  to 
achieve  standardization  in  the  computer  and  program 
area  as  a  serious  problem  or  not  is  not  important  to 
the  viewpoint  of  this  paper;  what  is  important  is  that 
it  be  recognized  as  a  concomitant  of  computer  and 
program  standardization.  It  is,  of  course,  obvious 
that  the  given  definition  of  standardization  does  not 
necessarily  convey  either  this  or  perhaps  other  tech¬ 
nical  concomitants  of  standardization.  These  kinds  of 
failures  of  definition  may  result  in  substantial  mis¬ 
understanding  of  the  price  one  must  pay  in  order  to 
achieve,  for  example,  standardization  as  defined. 

It  has  been  suggested  that  the  data  management 
capabilities  approach  to  command  control  systems 
development  reduces  the  problem  of  concomitant 
standardization  to  a  manageable  size.  The  problem 
here  is  that  this  prescription  does  not  inherently 
delimit  the  “kind”  of  command  control  system. 
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The  suggestion,  however,  does  seem  to  be  true  within 
limits:  (1)  the  kind  of  system;  (2)  the  inputs  are 
generally  a  result  of  human  sensing  of  the  environ¬ 
ment;  (3)  the  outputs  are  generally  directed  to  people 
for  further  consideration;  and  (4)  the  system  as  a 
whole  tends  to  be  nonreal  time  (e.g.,  few  predictable 
system  response  times  with  respect  to  its  environ¬ 
ment). 

Evolution 

Certainly  one  of  the  most  important  concepts  today 
in  developing  automated  information  systems  for  com¬ 
mand  control  is  that  of  evolution.  This  word  has  been 
used  to  describe  the  way  in  which  such  systems  are 
going  to  be  placed  into  the  operating  inventory  of  the 
Air  Force.  Evolution  seems  to  mean  beginning  with 
some  established  system  base  line  and  altering 
portions  of  that  base  line  through  time  so  as  to  pro¬ 
vide  needed  improvements  to  the  command  control 
system.  Examples  of  what  this  seems  to  mean  are 
that  there  may  be  changes  in  computers  but  in  such 
a  manner  that  does  not  require  radical  alterations  to 
the  communication  system;  the  upgrading  of  the 
communication  system  in  a  way  which  will  be  com¬ 
mensurate  with  the  computer  system;  adding  to  or 
modifying  the  man-machine  interface  in  a  way  which 
permits  continuity  in  the  staff  operation. 

Parenthetically,  with  regard  to  this  concept,  two 
comments  should  be  made:  (1)  Most  evolution  to  date 
has  been  confined  within  the  computer/program  sub¬ 
system;  (2)  it  has  been  in  the  semi-automated  air 
defense  environment  where  some  evolution  as  defined 
above  has  taken  place.  Certainly  the  original  system 
was  developed  in  a  way  which  was  not  consistent 
with  evolution  as  defined  above. 

Evolution  defined  in  this  fashion  recognizes,  but 
not  explicitly,  system  boundaries  broader  than  just 
the  computer.  On  the  other  hand,  acceptance  of  these 
broader  boundaries  raises  a  number  of  problems,  a 
most  important  question  being  evolution  from  what; 
that  is,  to  ask  what  is  a  minimal  base  line  from  which 
one  begins  to  evolve,  and,  for  a  given  base  line,  what 
is  a  reasonable  growth  expectancy? 

Evolution  was  defined,  more  or  less,  as  a  process  to 
be  applied  to  command  control  systems.  Once  again, 
the  operational  characteristics  of  the  process  have 
not  been  made  explicit  and  the  cases  (systems)  to 
which  the  concept  is  to  be  applied  are  not  well 
defined.  It  is  not  clear  whether  adequate  definition 
of  the  systems  would  be  sufficient  or  whether  the 
dimensions  of  evolution  need  also  be  established. 

That  some  sort  of  minimal  base  line  capability  is 
required  for  evolution  seems  obvious.  The  mini¬ 
mum  requirements  for  the  various  subsystems  have 


not  been  determined.  One  would  suspect  the  require¬ 
ments  would  vary  according  to  the  “kind”  of  system; 
that  is,  just  where  on  the  various  dimensions  of 
measurement  a  given  system  might  fall.  Certainly 
at  an  absolute  minimum,  all  of  the  various  subsystems 
ought  to  be  considered.  This  is  another  way  of  saying 
more  explicit  definition  of  command  control  system 
is  required  if  the  concept  of  evolution  is  even  to  be 
discussed  in  a  meaningful  way. 

An  example  of  the  kind  of  definition  and/or  con¬ 
sideration  required  is  that  of  the  Air  Force  Interim 
Command  Control  System  (AFICCS).  The  installa¬ 
tion  of  a  computer  in  a  command  control  environ¬ 
ment  at  a  major  air  command  raises  immediate  and 
significant  problems  of  the  relationship  of  that 
computer  to  the  existing  communication  network. 
When  a  communication  network  is  largely  manual, 
largely  manual  means  of  adapting  the  computer  to 
that  environment  are  used  as  the  solution.  This 
tends  to  force  update  and  maintenance  of  the  data 
base  into  off-line  batch  processing  mode  where  this 
means  substantial  human  intervention  between  the 
communication  subsystem  and  the  computer  program 
subsystem.  An  operating  command  which  is  attempt¬ 
ing  to  use  such  a  computer  in  support  of  command 
post  activities  such  as  monitoring  the  execution  of 
operations  is  faced  with  the  serious  problem  of 
trying  to  use  a  computer  operating  in  an  off-line  batch 
processing  mode  in  a  environment  in  which  rapid 
response  on-line  processing  is  generally  more  ap¬ 
propriate. 

Adequate  consideration  of  these  two  subsystems, 
computer  processing  and  communications,  is  criti¬ 
cally  important.  The  basic  design  of  the  computer 
program  system  will  be  radically  affected  by  the 
assumptions  one  makes  concerning  the  communica¬ 
tions  subsystem. 

In  addition  to  the  definition  problem,  in  considering 
evolution  in  any  system  or  for  any  system,  it  seems 
to  be  very  important  that  we  begin  to  develop  some 
kinds  of  understanding  of  what  are  the  concomitants 
of  any  given  change;  that  is,  if  we  change  some  sub¬ 
system,  what  are  the  impacts  within  that  subsystem 
and  what  are  the  impacts  between  that  and  other 
subsystems? 

In  order  to  better  understand  the  dynamics  of 
change,  it  is  necessary  to  have  better  definition 
so  that  concepts,  assumptions,  inferences,  etc., 
about  change  can  be  tested  and  modified  against 
the  reality  of  experience. 

Organic  operation 

One  of  the  strongest  trends  in  the  recent  past  has 
been  the  requirement  of  organic  software  capability 
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for  command  control  systems;  that  is,  the  “blue- 
suit”  or  “in-house”  computer  program  capability 
that  is  required  for  these  kinds  of  systems.  This 
capability  is  generally  thought  of  as  being  composed 
of  some  portion  each  of  Air  Force  and  civil  service 
personnel,  although  no  ratios  have  been  established. 
At  the  present  time  the  military  personnel  generally 
have  some  operational  Air  Force  Speciality  Code  with 
a  C  or  D  prefix  indicating  programming  or  systems 
analysis  capability. 

The  tasks  to  be  performed  by  the  in-house  capa¬ 
bility  have  not  been  well  defined;  consequently,  the 
types  and  skill  levels  have  not  been  adequately 
identified.  The  requirement  for  an  in-house  program¬ 
mer  and  analysis  capability  has  resulted  in  a  num¬ 
ber  of  problems  in  the  personnel  area.  For  example, 
the  installation  of  AFICCS  has  created  personnel 
problems  at  each  of  the  major  commands  possessing 
the  system.  Simply  stated,  there  are  not  enough  ex¬ 
perienced  programmers  and  analysts  to  provide 
adequate  timely  manning.  For  the  future,  moreover, 
rotation  and  reassignment  policies  for  Air  Force 
personnel  tend  to  mean  that  for  each  major  air 
command  at  any  given  time  one  third  of  their  per¬ 
sonnel  will  be  recent  arrivals  on  the  scene.  Thus, 
the  need  to  provide  for  improvement  of  individual 
and  group  performance  via  training  is  continual. 
This  need  re-emphasizes  the  need  for  adequate  task 
and  skill  level  definition. 

One  suggestion  to  alleviate  this  problem  is  to  enable 
transfer  of  experienced  individuals  amongst  the 
major  air  commands  possessing  this  kind  of  system. 
On  the  other  hand,  one  of  the  reasons  for  using 
operationally  experienced  personnel  with  the  C  or 
D  prefix  was  to  provide  the  computer  programming 
and  design  activity  (the  organic  capability)  with 
the  operational  experience  that  was  felt  critical 
to  appropriate  maintenance  and  development  of  any 
command  control  system.  Therefore,  if  it  were  to 
become  a  practice  to  transfer  command  control 
specialists  amongst  commands  in  order  to  retain 
system  continuity  and  competency,  there  is  an 
immediate  question  about  degradation  to  the  goal  of 
having  an  operationally  competent,  command-knowl¬ 
edgeable  person  dealing  with  the  command  control 
system  problem.  It  would  seem  that  a  practice  of 
transferring  command  control  specialists  from 
place  to  place  would  merely  alter  the  fracture  point 
between  the  command  control  information  proc¬ 
essing  specialists  and  the  operating  user,  not  nec¬ 
essarily  resolve  that  problem. 

It  has  been  argued  that  this  problem  is  reduced  by 
the  use  of  data  management  systems.  The  concept  of 
data  management  systems,  however,  raises  totally 


different  problems  in  the  personnel  area.  It  will  be 
recalled  that  data  management  systems  provide  a 
technique  by  which  nonprogrammers  can  do  those 
things  which  in  other  systems  are  done  by  program¬ 
mers.  As  these  kinds  of  systems  come  into  Air  Force 
use,  impact  on  personnel  must  be  considered.  To 
date  nearly  all  of  the  work  on  these  systems  has 
been  done  by  people  from  a  population  substantially 
different  from  that  made  up  by  Air  Force  staff 
officers.  It  is  not  obvious  that  all  of  the  successes 
reported  in  the  experimental  R&D  environment  are 
directly  translatable  to  the  operational  environment. 
Assume,  however,  that  such  systems  are  introduced 
either  as  the  result  of  additional  experimentation 
or  because  the  expected  gains  are  estimated  to  out¬ 
weigh  any  disadvantages.  It  would  appear  that  it 
would  be  quite  important  to  place  emphasis  on  training 
the  typical  staff  officer  to  use  such  systems.  This 
course  of  action  would  provide  for  greater  utility 
and  wider  acceptance  than  emphasis  on  training  for 
in-house  command  control  system  programming,  the 
more  usual  definition  of  organic  capability. 

The  foregoing  discussion  indicates  that  the  be¬ 
havioral  requirements  of  an  organic  capability  may 
well  be  subject  to  change  as  a  result  of  new  tech¬ 
nological  development.  It  would  be  desirable  under 
these  conditions  to  be  able  to  examine  in  detail  the 
impact  on  task  requirements  likely  to  be  brought 
about  by  introduction  of  data  management  systems. 
As  has  been  pointed  out,  this  is  not  possible;  thus, 
it  is  difficult  to  identify  the  changes  to  a  concept 
which  might  be  necessary  as  a  result  of  changes 
to  the  environment. 
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INTRODUCTION 

Some  preliminary  results  and  opinions  are  now  avail¬ 
able  based  on  experiences  in  building  and  using  the 
ADAM  system,  an  experimental  generalized  informa¬ 
tion  processing  system.  Although  not  comprehensive, 
these  conclusions  point  out  some  areas  of  eoneern  to 
developers  of  software  for  large  generalized  systems. 

In  particular,  experiences  with  ADAM  and  some 
of  its  users  indicate  that: 

Generalized  software  techniques  compare  favor¬ 
ably  with  more  conventional  methods  of  building 
large  program  systems  in  that  their  use  reduces  the 
time  required  to  develop  and  implement  application 
programs.  A  corresponding  cost  accrues  in  memory- 
space  or  operating-time  efficiency,  and  unless  the 
data  base  structures  and  procedure  definitions  are 
optimized  with  respect  to  the  system  and  the  job 
this  cost  may  be  unacceptable. 

For  effective  use  of  generalized  software  on  large 
problems,  the  user  must  control  the  optimization  of 
his  data  base  structure  and  procedure  definition.  He 
must  understand  enough  about  how  the  system  works 
to  exercise  this  control,  and  the  software  must  allow 
him  to  do  it.  In  other  words,  the  software  must 
provide  the  user  simple  ways  to  do  simple  things  and 
powerful  ways  to  do  complex  things — a  principle 
we  might  call  “graded  simplicity.” 

Some  individual  software  features  can  be  recom¬ 
mended  as  feasible  and  desirable  in  promoting  gen¬ 
eralization  or  helping  the  user  exercise  control.  Among 
theses  are: 

•  Extensive  capabilities  for  accepting  problem 
“hard-eodc,”  i.e.,  application  programs  written  in 
the  same  language  as,  and  to  the  same  interfaces 
as,  system  programs. 

•  A  provision  for  specifying  processes  in  a  proee- 
durc-oriented  language  as  opposed  to  and  in 
addition  to  a  command  or  query-oriented  lan¬ 
guage. 

•  A  provision  for  saving  individual  data  structures 
and  procedures  on  some  external  medium  (say 


magnetic  tape)  for  later  restoration  into  the 
same  or  another  version  of  the  system. 

•  The  capability  for  accepting  data  from  and 
generating  data  for  other  systems. 

Generalized  information  processing 

The  types  of  information  systems  to  which  these 
remarks  pertain  directly  are  those  concerned  with  the 
generation  and  maintenance  of  data  bases;  with  process¬ 
ing  these  data  to  reorganize,  query,  and  generate  output 
from  it;  and  with  performing  specified  calculations  on 
the  data.  These  information  systems  are  generalized  in 
that  they  isolate  the  programs  from  the  data  through 
separate  data  description  mechanisms  and  they  in¬ 
clude  programs  which  perform  common  functions  not 
specialized  to  any  single  application.  A  number  of 
such  systems  classified  by  terms  such  as  “data  man¬ 
agement  systems”  or  “generalized  information  systems” 
and  recently  proposed,  designed,  or  implemented  have 
been  described  (ESD,  1965;  IBM,  1965). 

The  ADAM  system  (Connors,  1966;  ADAM  Project, 
1966)  is  an  experimental  large-scale  generalized  in¬ 
formation  system,  designed  as  a  laboratory  tool  to 
perform  the  functions  described  above  (along  with 
many  others  such  as  display  handling  and  remote 
access).  ADAM  has  been  applied  on  an  experimental 
basis  to  implement  portions  of  a  number  of  different 
applications,  including: 

•  a  tactical  airborne  radar  system  characterized  by 
high  volume,  real-time  inputs; 

•  a  prescheduling  tool  for  satellite  missions  which 
emphasizes  man-machine  interaction  through  dis¬ 
play  consoles; 

•  a  personnel  assignment  problem  with  a  special¬ 
ized  algorithm  for  matching  men  to  jobs  and 

•  an  inventory  requirements  analysis  problem  with 
an  extremely  large  data  base. 

The  inventory  problem  was  by  far  the  largest  and 
longest.  It  was  conducted  in  conjunction  with  the  Air 
Force  Logistics  Command  (AFLC),  and  involved  gen¬ 
erating  files,  performing  calculations,  and  producing 
reports  which  duplicated  the  AFLC  Consumption  Item 


395 


396  Information  System  Science  and  Technology 


Requirements  Computation  (D041),  using  subsets  of 
their  170-million  character  data  base. 

These  applications  provide  the  foundation  for  the 
opinions  presented.  No  formal  quantitative  evaluation 
is  available  at  this  time;  however,  a  report  of  the  AFLC 
experiment  is  forthcoming  (Char  and  Foreman,  1966) 
and  an  evaluation  of  the  ADAM  system  is  planned. 

Generalized  software  effectiveness 

The  very  diversity  of  application  areas  to  which 
ADAM  has  been  applied  supports  the  viewpoint  that 
generalized  capabilities  can  be  provided.  An  experience 
with  one  part  of  the  AFLC  experiment  demonstrates  the 
effectiveness  of  generalized  techniques  as  compared 
with  conventional  programming.  The  segment  in  ques¬ 
tion  requires  15,000  Autocoder  instructions  in  the 
original  operational  system.  The  corresponding  nine 
major  procedures  comprise  approximately  55  state¬ 
ments  in  the  ADAM  file  manipulation  language.  No 
measure  of  the  time  and  effort  required  to  program 
the  original  is  available,  but  it  would  be  very  surprising 
if  it  were  less  than  the  five  man-months  it  took  to 
replicate  it  in  ADAM — complete  with  a  problem 
re-analysis,  since  the  problem-specification  documents 
were  not  available  to  us. 

The  file  manipulation  statements  were  not  simple — 
the  process  itself  was  quite  complex.  In  some  cases,  it 
took  a  long  time  to  write  an  appropriate  query  to 
retrieve  from  the  data.  But  a  query  could  be  written. 
A  user  reports,  .  .  only  simple  queries  were  answered 
in  a  time  span  representative  of  on-line  operation  but 
even  those  that  took  longer,  several  days  to  a  week, 
were  faster  than  any  other  method  that  could  have 
been  used  to  get  the  data,  if  it  could  be  obtained  at  all 
by  other  means”  (emphasis  supplied). 

On  the  other  hand,  users  react  to  the  complexity  of 
the  system  and  the  ease  with  which  it  is  possible  to 
cause  gross  inefficiencies  in  computer  use  with  ap¬ 
parently  simple  inputs.  More  about  that  in  the  following 
section. 

User  involvement 

Perhaps  the  most  important  conclusion  reached 
through  the  ADAM  experience  is  that  the  user 
of  a  generalized  system  must  become  involved  in 
it — to  the  degree  that  he  understands  it  well  enough  to 
specify,  in  the  neeessary  detail,  the  processes  he  initiates. 
The  requirements  then  arc  three:  the  user  must  be 
aware  of  the  implications  of  how  his  data  is  structured; 
he  must  understand  and  control  the  optimization  of 
his  procedures;  and  the  system  must  allow  control  of 
optimization. 

For  a  trivial  but  real  example,  in  one  application  the 
user  actually  needed  to  perform  the  supposedly  simple 


process  described  in  almost  every  elementary  computer 
programming  text:  multiply  price  by  quantity  to  get 
total  cost.  Due  to  the  way  his  data  were  organized 
when  received,  priee  and  quantity  were  in  different  files, 
with  corresponding  entries  linked  by  a  symbolic  iden¬ 
tifier.  The  ADAM  file  generation  statements  generated 
the  file  in  the  order  of  the  original  data,  for  simplicity. 
The  single  statement  whieh  accessed  the  data,  per¬ 
formed  the  multiplication,  and  stored  the  result  was 
simple  to  write,  even  on-line.  The  resulting  processing 
involved  serially  accessing  the  file  which  contained 
price,  and,  for  each  entry,  directly  accessing  the  entry 
in  the  other  file  which  contained  quantity.  In  the  ADAM 
system,  direct  access  is  slower.  Although  the  simplicity 
of  the  query  suggested  a  fast  response,  the  processing 
actually  went  on  interminably  until  it  was  prematurely 
stopped  after  one  hour’s  running  time.  Later  analysis 
estimated  that  it  would  take  two  hours  to  complete. 

By  moving  price  and  quantity  to  the  same  file,  the 
user  reduced  the  running  time  required  by  a  factor  of 
eight.  He  preferred  a  faster  calculation  at  the  cost  of 
a  somewhat  more  cumbersome  file  generation.  But  only 
he  could  tell  if  changing  the  structure  in  this  way  would 
meet  his  requirements  for  other  uses  of  price  and 
quantity.  Different  processes  he  specified  could  turn 
out  to  be  much  slower  because  price  was  no  longer 
in  its  original  file.  Even  if  he  decided  to  keep  price  in 
both  files,  he  would  not  solve  his  optimizing  problem. 
Two  copies  in  two  files  implies  a  more  complicated 
and  time-consuming  update  process  when  prices  change. 

This  experience  and  a  number  of  similar  experiences 
serve  as  convincing  evidence  that  a  system  can  help 
a  user  optimize  by  giving  him  facilities  to  restructure 
his  data,  but  it  cannot  optimize  for  him.  And  he  cannot 
optimize  unless  he  knows  how  the  system  will  perform 
the  processes  he  specifies. 

Consider  another  example.  In  the  ADAM  system, 
the  statement  which  begins 

FOR  CITY  BOSTON  . . . 

causes  a  direct  access  to  the  BOSTON  entry  in  the 
CITY  file.  The  apparently  equivalent  statement: 

FOR  CITY  .  IF  NAME  EQ  BOSTON  .  .  . 
causes  a  serial  search  through  entries  in  the  CITY  file 
until  BOSTON  is  found.  The  user  must  know  and  ap¬ 
preciate  the  difference  in  order  to  query  the  system 
effectively. 

One  way  to  avoid  requiring  a  user  to  understand  his 
software  is  to  devise  a  system  with  only  one  way  to 
do  an  operation — a  form  of  uniform  simplicity.  Some 
systems,  for  example,  provide  direct  access  through  an 
index  to  all  data  items.  This  forces  simplicity  for  the 
user,  but  costs  him  effectiveness.  For  example,  a 
fully  indexed  data  structure  can  provide  direct  access 
to  every  data  item.  If  a  system  always  requires  this, 
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it  may  spend  a  prohibitive  amount  of  computer  time 
making  indices  for  data  items  which  a  user  always 
accesses  serially. 

Uniform  simplicity  hides  a  pitfall.  The  easy  avail¬ 
ability  of  uniformly  simple  ways  to  initiate  processes 
deceives  a  user  of  information  processing  systems  when 
it  convinces  him  that  his  problem  automatically  be¬ 
comes  simple. 

Graded  simplicity  is  required.  For  truly  simple  tasks, 
simple  specifications  are  sufficient  and  should  be  avail¬ 
able.  For  tasks  with  more  complex  elements,  ways 
to  make  complex  but  efficient  specifications  are  es¬ 
sential.  Our  experience  shows  most  real  jobs  contain 
both  kinds  of  tasks. 

Specific  software  features 

The  following  recommendations  for  specific  software 
features  are  not  intended  to  be  comprehensive,  but 
represent  some  suggestions  derived  from  ADAM  ex¬ 
perience. 

Procedure  language — In  the  ADAM  file  handling 
language,  the  actual  operations  performed  as  the  result 
of  a  message  are  implicit  in  query-type  statements — for 
example,  the  user  cannot  explicitly  control  iterations 
within  several  files.  On  this  account,  he  loses  an  amount 
of  control  over  optimization  of  complex  processes.  A 
language  that  allows  him  to  describe  processing  pro¬ 
cedures  (in  addition  to  commands  and  simple  queries) 
is  required. 

Hard-code  interface — In  the  final  analysis,  a  system 
is  truly  generalized  only  if  it  provides  for  the  inclusion 
of  specialized  user-coded  programs  treated  similarly  or 
equivalently  to  system  programs.  The  mechanism  must 
provide  not  only  for  problem  programs  to  be  run  as 
separate  tasks  of  their  own,  but  for  them  to  be  run  in 
place  of,  or  along  with,  system  functions. 

For  example,  ADAM  provides  for  a  user  to  specify 
conversion  routines  to  be  applied  automatically  to 
every  input  and  output  of  a  given  item  of  data.  Some 
system  routines  are  available,  but  provision  is  made 
for  a  user  to  include  his  own  and  to  prespeeify  where 
they  are  to  be  applied.  Similarly,  in  the  ADAM  report- 
and  output-formatting  definition  mechanism,  an  op¬ 
eration  is  available  (in  addition  to  those  like  “space 
page,”  “print  value,”  etc.)  which  specifies  that  a 
routine  is  to  be  executed  at  this  point  in  the  output 
formatting  process.  These  provisions  were  liberally 
used  in  certain  applications;  more  of  them  would  have 
been  desirable,  for  example  a  provision  for  user  hard- 
code  at  message  input  time. 

An  effective  capability  for  accepting  hard-code  re¬ 
quires  a  well-defined  interface  with  system  routines 
which  allows  access  not  only  to  the  data  base,  but  to 
volatile  data  such  as  lines  of  report  just  before  they 


are  output  and  parts  of  messages  just  after  they  are 
input.  And  in  addition  to  explicit  calls  on  these  routines, 
the  provision  must  exist  to  have  them  called  implicitly  at 
predefined  points  during  the  operation  of  a  system 
function. 

Saving  and  restoring — Many  data  base  systems,  in¬ 
cluding  ADAM,  treat  the  system  programs  and  data  as 
part  of  the  data  base  and  go  through  a  more  or  less 
elaborate  process  of  allocation  and  transformation  in 
absorbing  new  data  or  programs.  During  ADAM  use, 
a  requirement  that  became  obvious  was  to  allow  for 
saving  individual  data  structures  and  routines  outside 
the  system  (ADAM  does  it  by  writing  them  on  a  mag¬ 
netic  tape)  so  that  parts  of  the  system  may  be  changed 
or  replaced  without  requiring  regeneration  of  all  the 
rest;  instead,  the  saved  structures  may  be  restored  to 
the  same  system  or  to  a  different  version. 

From  this  saving  and  restoring  capability,  three  im¬ 
mediate  benefits  accrue: 

•  The  data  and  programs  for  a  single  application 
may  be  transferred  from  a  current  version  of  a 
system  to  a  different,  presumably  improved 
version  thereby  separating  the  maintenance  of  sys¬ 
tem  from  that  of  application; 

•  two  applications  which  may  not  be  able  to  exist 
together  in  the  system  (perhaps  together  they 
exceed  the  system  capacity)  can  each  run  with 
exaetly  the  same  system  software;  and 

•  relatively  infrequently  used  system  data  and 
programs  need  not  be  kept  as  a  permanent  part 
of  the  system. 

Accept  and  generate  outside  data — Experience  shows 
that  data  more  frequently  eome  from  and  go  to  other 
computer  programs  or  systems;  less  frequently  are  pre¬ 
pared  especially  for  a  single  system.  At  one  time  the 
ADAM  file  generation  process  made  the  mistake  of 
expecting  all  variable  length  input  fields  to  be  termi¬ 
nated  by  a  (user-speeified)  character.  Data  do  not  nec¬ 
essarily  eome  in  that  way,  and  in  several  applications  a 
separate  preprocessing  step  was  required  to  insert  the 
terminators.  A  generalized  system  must  expect  to 
aeeept  and  generate  data  not  specifically  prepared  for 
it. 

CONCLUSION 

Generalized  software  techniques  for  advanced  informa¬ 
tion  processing  systems  are  with  us  at  least  experimen¬ 
tally,  and  point  the  way  to  making  large  problems 
somewhat  more  tractable.  Nevertheless,  there  is  no 
magic  to  them  and  it  is  clear  that  the  user  of  a  system 
cannot  be  decoupled  enough  from  the  way  his  problem 
is  solved  to  avoid  understanding  and  controlling  ef¬ 
fective  optimization.  As  results  of  experiences  with 
experimental  software  techniques  become  available,  they 
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will  point  the  way  to  acceptable  balances  between  sim¬ 
plicity  and  effectiveness. 
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INTRODUCTION 

The  year  1966  may  well  go  down  in  the  annals  of  the 
military  data  automation  community  as  the  Year  of  the 
Master  Plans.  Comforting  as  this  thought  may  be — for 
it  is  helpful  to  have  a  clear  blueprint  of  the  tasks  the 
future  holds — it  is  not  clear  that  1966  will  also  be  known 
as  the  Year  of  the  Reconciliation  of  the  Master  Plans. 
And  until  such  coordination  of  master  plans  is  effected 
within  the  military  services,  throughout  the  Depart¬ 
ment  of  Defense,  and,  yes,  possibly  across  the  entire 
structure  of  the  U.S.  Government,  it  appears  unlikely 
that  the  master  plans  of  1966  can  hope  to  serve  as 
blueprints  for  more  than  a  year  or  two  of  the  future.  Co¬ 
ordination  of  master  plans  is  not  easily  effected,  and 
because  of  the  potential  controversy  involved,  the  sub¬ 
ject  is  not  to  be  lightly  broached.  But  it  has  been  dem¬ 
onstrated  many  times  that  once  government  spending 
for  a  single  identifiable  function  or  item  or  utility 
becomes  a  substantial  percentage  of  the  budget,  more 
cooperation  and  less  duplication  of  effort  is  demanded 
on  the  part  of  all  agencies  involved.  As  a  typical  ex¬ 
ample,  communications  services — first  within  the  DOD 
and  now  within  the  government  in  toto — are  coming 
under  increasingly  greater  scrutiny  to  insure  economy 
and  efficiency  of  service  and  operation.  Government 
interest  in  close  supervision  of  all  of  its  data  automation 
facilities  and  services  is  either  entering  or  will  soon 
enter  this  same  phase.  The  Bureau  of  the  Budget’s 
yearly  publication,  Inventory  of  Automatic  Data  Pro - 
cessing  Equipment  in  the  Federal  Government ,  is  a  first 
step  in  that  direction. 

Each  organization  finds  a  master  plan  for  the  future 
a  necessary  and  useful  tool  for  internal  planning  pur¬ 

*Any  views  expressed  in  this  paper  are  those  of  the  author. 
They  should  not  be  interpreted  as  reflecting  the  views  of  The 
RAND  Corporation  or  the  official  opinion  or  policy  of  any 
of  its  government  or  private  research  sponsors. 


poses,  but  coordination  of  the  master  plans  of  several 
different  organizations — each  plan  separately  conceived 
— may  be  a  painful  and  trying  experience  for  all  con¬ 
cerned.  However,  successful  coordination  of  planning 
efforts  can  provide  much  greater  stability  and  effective¬ 
ness  of  the  composite  of  efforts.  This  paper  will  not 
proffer  suggestions  that  will  necessarily  ease  the  burden 
of  data  automation  master  plan  coordination,  but,  be¬ 
cause  it  is  intended  not  as  a  presentation,  but  as  a 
vehicle  to  stimulate  discussion,  the  paper  will  attempt 
to  raise  some  of  the  broad  issues  that  at  all  levels  face 
the  designers  of  master  plans,  or  plans  that  coordinate 
master  plans.  The  treatment  of  these  issues  in  the  paper 
will  be  kept  as  objective  as  possible;  it  will  be  left  to 
the  discussants  at  the  Third  Congress  to  take  sides  and 
do  battle. 

To  achieve  the  goal  of  delineating  some  future  trends 
of  military  data  processing  and  to  indicate  some  of  the 
potential  effects  that  masted  planning  and  the  coordi¬ 
nation  of  master  plans  might  have  on  these  trends,  in 
the  remainder  of  this  paper  I  will  first  undertake  to 
categorize  the  functional  uses  of  data  processing  by 
the  military;  next,  the  relative  scope  of  data  automation 
activities  by  the  services  will  be  outlined;  third,  a  few 
of  the  dichotomies  (or  paradoxes?)  facing  master  plan¬ 
ners  will  be  discussed;  then  some  of  the  potential  bottle¬ 
necks  that  may  slow  the  desired  progress  of  data 
automation  improvement  will  be  listed;  and  finally, 
some  summarized  suggestions  of  future  planning  efforts 
will  be  presented. 

Major  military  uses  of  data  automation 

In  the  military,  as  elsewhere,  the  user  of  data  auto¬ 
mation  sees  its  greatest  utility  and  future  as  an  aid  to 
him  in  his  work.  Understandably,  the  user  frequently 
thinks  that  he  is  making  the  very  best  use  of  his  machine 
and  that  improvements  and  advances  stem  foremost 
from  his  efforts.  And  the  user’s  data  processing  equip¬ 
ment  often  is  sufficiently  flexible  to  allow  the  user  to 
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branch  into  activities  similar  to  those  of  other  organiza¬ 
tions  or  agencies;  this  has  a  tendency  to  engender 
competition  and  rivalry.  Thus,  in  categorizing  the  uses 
to  which  the  military  applies  data  automation,  it  is 
recognized  that  some  overlap  will  exist  and  that  these 
overlaps  may  fall  in  areas  where  today  competition  and 
rivalry  may  exist.  For  the  purposes  of  this  paper,  five 
major  military  uses  of  data  automation  are  considered, 
although  it  is  quite  likely  that  others  could  be  cited; 

•  Research  and  planning  systems 

•  Management  systems 

•  Support  systems 

•  Command  systems 

•  Tactical  systems* 

Research  and  planning  data  automation  systems  in¬ 
clude  those  at  military  laboratories  and,  typically,  the 
system  recently  proposed  for  use  by  the  Air  Staff  for 
preparation  of  plans  and  the  formulation  of  staff  posi¬ 
tions.  Ultimately,  the  research  systems  at  the  lab¬ 
oratories  may  be  widely  internetted;  a  plans  system  for 
the  Air  Staff  might  have,  in  addition  to  an  information 
display  center,  remote  connections  to  the  deputy  chiefs 
of  staff  and  appropriate  directorates,  as  well  as  connec¬ 
tions  that  make  available  information  from  the  data 
bases  of  other  systems. 

Management  systems  are  taken  to  be  those  that  re- 
curringly  perform  essentially  the  same  tasks,  whether 
in  peace  or  crisis,  that  arc  necessary  for  the  normal 
functioning  of  a  large  organization,  viz.,  finance  and 
accounting,  personnel  records,  logistics,  etc.  Generally, 
elements  of  branches  of  such  systems  are  to  be  found 
at  every  military  installation. 

Support  systems  arc  typified  by  those  in  use  by  the 
Air  Weather  Service,  the  photocomposing  system  for 
document  publication  soon  to  be  installed  at  Wright- 
Patterson  Air  Force  Base,  etc.  Support  systems  charac¬ 
teristically  have  one  or  a  very  few  large  data  processing 
facilities  and  a  very  large  number  of  users  of  their 
output. 

Command  systems  have  sometimes  been  described 
as  “capping”  systems,  for  they  are  apt  to  make  use  of 
the  outputs  of  planning  systems,  management  systems, 
and  support  systems.  In  the  future,  as  tactical  computer 
systems  become  more  prominent,  upper  echelon  com¬ 
mand  systems  will  likely  make  real-time  use  of  sum¬ 
marized  data  from  them.  In  a  sense  command  systems 
either  are,  or  should  be,  capable  of  mustering,  allocating, 
and  directing  resources  to  meet  military  commitments — 
both  potential  and  actual — throughout  all  levels  of 
crisis  and  war.  Of  course,  this  broad  charter  places 
the  command  system  in  the  position  of  overlapping 

♦Special  purposes  computers,  such  as  those  used  in  missiles, 
are  not  included. 


the  functional  areas  of  planning,  management,  and 
(sometimes)  support.  And,  with  the  present  worldwide 
politico-military  environment  directed  toward  controlled 
escalatory  warfare,  it  is  inevitable  that  even  upper 
echelon  command  systems  will  occasionally  encroach  on 
some  control  functions  of  tactical  systems. 

It  is,  of  course,  impossible  to  avoid  functional  overlap 
in  establishing  a  category  of  tactical  data  automation 
systems.  It  may  be  desirable  to  categorize  tactical  sys¬ 
tems  as  those  in  use  by  field  forces  (whether  in  the 
field  or  in  garrison  training),  or  to  say  that  tactical 
systems  are  mobile,  ruggedized,  and,  hopefully,  small 
and  lightweight.  But  probably  the  best  differentiation 
possible  is  to  say  that  tactical  systems  arc  those  used 
by  field  forces,  and  not  already  covered  by  one  of  the 
other  four  categories.  Thus,  elements  of  SAGE  and 
BUIC,  shipborne  and  airborne  systems,  as  well  as  units 
that  may  be  carried  into  combat,  such  as  the  Field  Ar¬ 
tillery  Digital  Automatic  Computer  (FADAC),  are 
included.  In  general,  data  processors  that  arc  integral 
parts  of  weapon  systems  and  are  essentially  special 
purpose  (e.g.,  inertial  navigation  computers)  arc  not 
included  here. 

Is  it  necessary  to  have  this  categorization  of  functional 
uses  of  data  processing  by  the  military?  The  answer  is 
“Yes,”  and  for  at  least  two  reasons.  The  first  reason  is 
that  the  attempt  at  categorization  points  up  the  extreme 
difficulty  of  looking  in  a  meaningful  fashion  at  just  one 
part  of  the  military  data  automation  picture.  With  rare 
exceptions  it  is  simply  not  possible  (or,  at  least,  not 
effective  or  efficient)  to  consider  one  category  of  mili¬ 
tary  application  of  data  automation  (e.g.,  command 
systems)  without  being  cognizant  of,  and  coordinating 
with,  activities  in  most  of  the  other  areas.  Certainly  this 
is  true  for  many  tactical  applications,  for  often  they 
cither  arc  now,  or  in  the  future  will  be,  primarily  dupli¬ 
cations  of  efforts  that  are  already  handled  by  data 
processing  systems  that  are  not  capable  of  field  deploy¬ 
ment,  e.g.,  the  possible  future  use  of  data  processors  in 
backup  airborne  command  posts. 

A  second  reason  for  some  form  of  categorization 
stems  from  the  needs  of  higher  echelon  organizations  to 
delineate  similar  characteristics  for  comparison  of  the 
efficient  use  of  data  processing  systems.  For  example, 
the  DOD  must  at  some  point  view  the  use  of  data  auto¬ 
mation  by  the  three  services  in  some  comparative 
fashion,  asking  the  question,  “Are  systems  of  compar¬ 
able  capability  being  utilized  with  equal  effectiveness?” 
The  measures  of  utilization  may  be  difficult  to  establish, 
but,  once  established,  should  the  answer  in  any  instance 
be  negative,  remedial  action  would  be  needed.  In  the 
same  manner,  an  appropriate  organization  looking 
broadly  across  all  U.S.  Government  uses  of  data  auto- 
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mation  for  the  purposes  of  coordination  leading  to 
efficient  employment  of  data  automation  throughout 
all  government  agencies  must  also  seek  some  means  of 
categorization  for  comparison  and,  hopefully  coordi¬ 
nation. 

The  relative  scope  of  military  uses 

of  data  automation 

What  is  the  relative  effort  that  is  invested  in  pro¬ 
viding  military  data  automation?  Some  statistics  on 
this  question  may  help  to  provide  some  insight  into 
future  trends  of  military  data  processing.  The  estimated 
total  number  of  installed  computers  operating  under 
the  aegis  of  the  U.S.  Government  at  the  end  of  FY  66 
was  about  2500  (BOB  1965).  This  represents  about  9 
per  cent  of  the  total  number  of  computer  installations 
in  the  country  at  that  time.  Extrapolating  available  data 
to  the  present,  the  DOD  makes  use  of  about  1900  com¬ 
puters,  of  which  about  53  per  cent  are  under  control 
of  the  Air  Force,  with  the  Army  and  Navy  controlling 
22  per  cent  and  21  per  cent,  respectively.*  The  direct 
cost  of  data  automation  (with  support)  to  the  U.S. 
Government  has  been  over  $1  billion  for  each  of  the 
last  three  fiscal  years.  Thus,  about  one  per  cent  of  the 
national  budget  is  directly  attributable  to  government 
data  automation.  The  DOD  accounts  for  about  two- 
thirds  of  the  U.S.  Government's  installed  computers 
and  about  61  per  cent  of  its  computer  costs.  Some  of 
these  relationships  are  illustrated  in  Figure  1,  which 
shows  the  recent  growth  of  installed  and  on-order 
computers  in  the  U.S.,  installed  computers  in  foreign 
countries,  and  installed  computers  under  the  control 
of  the  U.S.  Government,  the  Air  Force,  Army,  and 
Navy. 

It  is  noteworthy  that  while  this  nation  has  been  in¬ 
stalling  additional  computers  at  the  rate  of  7,250  per 
year  for  the  past  two  years,  the  U.S.  Government  rate 
has  been  only  about  300  additional  computers  per  year, 
with  the  Air  Force  accounting  for  slightly  more  than 
one-third  of  the  new  additions  each  year  and  the  Army 
and  Navy  each  accounting  for  about  one-sixth  or  less 
of  the  total  government  rate.  This  should  probably  not 
be  surprising,  for  the  military  services  had  priority  in 
filling  their  needs  during  the  early  years  of  data  auto¬ 
mation  growth,  and  while  computer  replacements  con¬ 
tinue  apace  in  military  installations,  the  rate  of  addition¬ 
al  computer  acquisition  is  not  in  keeping  with  that  of  the 
nation. 

It  has  often  been  said  recently  that  the  impact  of 

*The  data  presenied  here  include  some  tactical  computer  instal¬ 
lations  such  as  SAGE  and  HU  1C  sites.  They  do  not  include 
airborne  or  shipborne  computers  or  field  mobile  units  such 
as  FADAC. 


government — and  especially  military — spending  on  the 
computer  industry  is  continually  dwindling.  Figure  1 
might  be  construed  to  substantiate  that  claim,  for  at  end 
of  FY  66  the  U.S.  Government  will  operate  less  than 
7  per  cent  of  the  nation's  installed  computers;  two  years 
previously  it  operated  over  9  per  cent.  Should  present 
trends  continue  for  the  next  two  years,  the  U.S.  Govern¬ 
ment's  share  will  drop  to  about  5.5  per  cent. 

In  broad  geographical  terms,  where  do  the  military 
services  make  use  of  data  automation?  Table  1  shows 
the  number  of  major  geographical  locations  of  military 
data  automation  locations  (BOB  1965). 

Table  I 


Distribution  of  major  locations  of  U.S.  military  data  processing 


Military 

Service 

Major  U.S. 
Urban  Areas 

Foreign 

Countries 

Total 

Air  Force 

126 

17 

143 

Army 

75 

4 

79 

Navy 

46 

4 

50 

Within  the  U.S.,  the  regions  with  the  highest  density 
of  military  computers  are  Washington,  D.C.,  San  Fran¬ 
cisco,  San  Antonio,  Philadelphia,  Norfolk,  Virginia, 
and  Dayton,  Ohio.  It  follows  that  each  of  these  regions 
would  be  a  prime  candidate  for  the  on-line  service  that 
may  someday  be  provided  by  a  data  automation  ‘‘utility" 
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system,  i.e.,  a  system  that  might  supply  on-line  com¬ 
puting  power  from  a  central  facility  to  all  military  cus¬ 
tomers  within  a  specified  geographical  area.  As  an 
aid  to  visualizing  the  potential  scope  of  military  data 
automation  utility  systems,  Table  II  indicates  the 
degree  of  gross  collocation  of  military  computers  within 
the  U.S.  It  is  stressed  that  collocation  is  taken  here  to 
mean  computer  installations  within  a  general  urban 
area,  i.e.,  within  at  least  a  few  tens  of  miles  of  each 
other. 


Table  II 

Distribution  of  collocated  military  data  processing  in  lhe  U.S. 


Military  Service 

Major  U.S. 

Combinations 

Urban  Areas 

Air  Force-Army-Navy 

9 

Air  Force-Army  (only) 

21 

Air  Force-Navy  (only) 

11 

Army-Navy  (only) 

4 

Table  II  indicates  that  possibly  one-third  of  the  Air 
Force’s  data  automation  U.S.  locations  might  be  able 
to  share  a  local  area  military  data  automation  utility 
system  with  one  or  more  of  the  other  services.  Should 
it  be  possible  to  serve  a  large  geographical  area  with 
the  utility,  obviously  even  more  installations  could 
be  supported.  Of  course,  data  automation  utility  sys¬ 
tems  present  certain  problems  and  drawbacks  when 
applied  to  military  tasks,  but  indications  are  that  com¬ 
mercial  applications  of  these  systems  will  become  more 
prolific  in  the  future  and  it  is  likely  that  thev  military  will 
find  it  necessary  to  evaluate  at  least  the  potential  use 
of  utility  systems,  both  for  individual  service  applications 
and  for  joint  service  use.  More  will  be  said  about  this 
later. 

The  discussion  on  scope  of  military  data  automation 
thus  far  has  centered  on  fixed  computer  installations, 
thus  excluding  most  tactical  applications.  While  Figure 
1  makes  evident  that  the  growth  of  fixed  military  com¬ 
puter  installations  is  moving  ahead  at  a  relatively 
moderate  pace,  it  gives  no  indication  of  the  future 
growth  of  the  use  of  data  automation  for  tactical  pur¬ 
poses.* 

While  it  is  doubtless  true  that  no  accurate  estimates 
can  be  made  of  the  degree  to  which  military  data  auto¬ 
mation  may  ultimately  be  applied  in  tactical  units,  it 
is  perhaps  of  some  value  to  make  a  gross  reckoning  of 
the  number  of  identifiable  military  units  to  which  data 
automation  may  be  applied  in  the  future.  An  estimate 
of  this  kind  is  given  in  Table  III,  which  also  indicates 

♦The  computers  considered  for  tactical  purposes  are  assumed 
to  be  of  the  micro-miniaturized  general  purpose  variety, 
costing  possibly  $30,000  to  $150,000  today.  Although  no  one 
of  these  computers  would  satisfy  all  tactical  needs,  it  is  as¬ 
sumed  that  a  computer  of  this  type  could  be  of  great  use 
in  many  lactical  applications,  if  appropriate  peripheral  equip¬ 
ment  and  software  could  be  made  available. 


a  range  in  the  number  of  computers  that  might  actually 
be  involved.  To  avoid  security  difficulties,  U.S.  force 
size  has  been  based  on  an  unclassified  source.  (ISS  64) 
Table  III 


Potential  future  use  of  tactical  military  computers 


Military 
Service 
or  Branch 

Unit 

No. 

of 

Units 

Tactical 
Computers 
per  Unil 

Total 

Air  Force 

A/C  Sqdn 

200-250 

1-2 

200-500 

Army 

Division 

16-19 

20* 

320-380 

Navy 

Ship/Sub 

400-500 

1-2 

400-900 

A/C  Sqdn 

60-70 

1-2 

60-140 

Marine  Corps 

Division 

3-4 

20 

60-80 

A/C  Sqdn 

15-20 

1-2 

15-40 

Grand  Total 

1055-2040 

♦Already  acquired  FADAC  computers  not  included. 


How  reasonable  are  the  totals  shown  in  Table  III? 
That  question  is  essentially  impossible  to  answer  and 
the  totals  can  be  considered  only  as  opinion,  but  some  of 
the  assumptions  underlying  the  totals  and  some  of  the 
implications  of  them  can  be  discussed.  The  use  of 
military  data  processing  in  tactical  environments  has 
long  been  stated  as  a  military  requirement  by  the  serv¬ 
ices.  However,  only  in  recent  years  has  the  state  of 
technology  permitted  the  production  of  data  processors 
sufficiently  small  and  reliable  to  be  seriously  considered 
for  field  use.  Acceptable  peripheral  equipment  to  work 
with  the  central  processing  unit  and  appropriate  soft¬ 
ware  are  today  probably  the  pacing  items  that  delay 
system  applications.  Of  course,  not  all  so-called  tactical 
applications  represent  the  worst  of  all  possible  opera¬ 
tional  environments;  for  example,  many  shipboard  a- 
plications  and  ground  applications  associated  with  air¬ 
craft  squadrons  may  present  much  more  benign 
environments  than  can  be  expected  for  equipment 
taken  by  Army  and  Marine  units  into  active  ground 
combat  zones.  Implicit  in  some  of  these  comments  is 
the  assumption  that  it  will  be  desirable  (and,  hence, 
required)  to  make  identifiable  military  fighting  units 
down  to  at  least  the  division,  ship,  and  squadron  level 
essentially  independent  of  higher  echelons  for  at  least 
a  major  portion  of  their  tactical  data  automation  cap¬ 
ability.  It  is  possible  that  the  successful  demonstration 
of  a  field  service  tactical  data  automation  utility  system 
could  reverse  this  assumption  and,  rather  than  organic 
computers,  the  computing  power  of  large  central  proces¬ 
sors  would  be  used  remotely  by  lower  echelon  units. 
The  mobile  communications  netting  task,  while  not 
technically  infeasible,  would  be  formidable  for  a  tactical 
utility  system.  In  general,  the  need  to  maintain  autono¬ 
mous  capability  in  the  operation  of  field  units  will  likely 
keep  the  pressure  high  for  separate  computers. 

In  terms  of  the  earlier  categorization  of  the  use  of 
military  data  processing,  it  would  appear  that  tactical 
uses  could  easily  duplicate  all  of  the  other  categories 
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listed  above.  (Should  enough  computer  power  be 
available  in  the  field,  it  is  to  be  anticipated  that  some 
of  it  would  be  relegated  to  certain  operations  research 
functions.)  Not  only  that,  but  it  is  likely  that  in  some 
instances  command  and  control,  planning,  management, 
and  support  functions  will  all  be  carried  out  in  the 
same  data  automation  installation,  e.g.,  aboard  ship, 
or  organic  to  the  tasks  of  the  aircraft  squadron. 

To  some  degree,  the  data  processor  organic  to  the 
lower  echelon  tactical  unit  may  itself  be  the  central 
element  of  a  small  remote  I/O  tactical  system,  for  the 
success  of  application  of  data  automation  to  tactical 
units  may  well  hinge  on  the  ability  to  acquire  informa¬ 
tion  remotely  in  digital  form,  process  the  information, 
and  send  processed  results  or  further  queries  to  lower 
or  higher  echelon  units.  The  successful  completion  of 
recent  tests  employing  a  digital  message  entry  and  ac¬ 
knowledgement  device  at  lower  echelons,  and  buffered 
and  printed  output  at  higher  echelons,  has  indicated 
the  advantages  and  feasibility  of  some  elements  of  a 
small  tactical  system  with  remote  input/output.*  It 
remains  to  be  demonstrated  that  a  central  processor 
unit  and  appropriate  software  can  be  added  to  com¬ 
plete  this  application  of  data  automation  to  one  part 
of  a  tactical  air  control  system.  If  a  complete  system 
can  be  made  reliable  and  successfully  demonstrated  in 
the  field,  similar  applications  in  other  areas  and  by 
other  services  will  likely  depend  more  on  the  generation 
of  software  than  on  equipment. 

Pitfalls  and  dichotomies  in  planning  for  military 

data  automation  systems 

The  road  to  be  followed  by  planners  of  future  military 
data  automation  systems  promises  to  be  a  rocky  one. 
The  requirements  for  future  military  data  automation 
systems  will  likely  continue  to  include,  to  some  degree, 
the  long-standing  ones  that  sometimes  seem  almost 
paradoxical:  efficient  and  economical,  evolutionary, 
flexible,  and  survivable.  These  requirements,  coupled 
with  the  great  geographical  coverage — often  worldwide 
— required  of  many  systems,  pose  systems  planning 
tasks  that  are  indeed  formidable.  The  recently  acquired 
third-generation  hardware  capabilities  have  created 
further  problems  for  the  planner  by  affording  various 
alternatives  that  may  be  employed  to  satisfy  military 
systems  requirements;  one  pair  of  alternatives  was 
implicitly  mentioned  above  in  the  discussion  of  tactical 
data  automation  and  is  discussed  in  more  detail  below. 

Utility  systems:  Yes,  no,  or  how  much? 

Probably  the  biggest  single  question  facing  the  plan¬ 

*Tests  consisted  of  sending  and  acknowledging  forward  air 
controller  messages  and  were  conducted  at  the  Tactical  Air 
Warfare  Center,  Eglin  Air  Force  Base,  Florida. 


ner  of  future  systems  concerns  the  choice  of  using  a 
utility  system  with  a  large  central  data  processor  and 
remote  input/output  or  of  using  several  smaller  (but 
not  necessarily  “small”)  data  processing  installations 
at  each  of  the  major  I/O  locations.  At  the  moment 
there  are  no  good  guidelines  available  to  help  the 
planner  in  making  this  choice,  for  even  special  purpose 
utility  systems,  such  as  the  Keydata  Corporation  system 
in  the  Boston  area,  are  just  beginning  to  provide 
operational  information  (C&A,  January  1966).  A 
corollary  question  facing  the  planner  concerns  the  size 
of  geographical  coverage  to  be  provided  by  a  utility 
system;  this  is  closely  associated  with  the  further  ques¬ 
tion:  should  the  utility  system  provide  capability  to 
more  than  one  military  service  within  a  given  geo¬ 
graphical  area?  This  question  has  been  broached  earlier 
in  this  paper,  when  it  was  pointed  out  that  there  are 
certain  areas  in  the  U.S.  where  a  heavy  concentration 
of  military  data  automation  is  to  be  found.  Under 
selected  circumstances — notably,  those  associated  with 
universities  and  colleges — early  models  of  utility  sys¬ 
tems  appear  to  be  working  well  and  the  users  seem 
to  be  more  satisfied  than  not.  In  installations  at  institu¬ 
tions  of  higher  learning,  the  utility  system  is  often 
used  partially  in  a  time-shared  mode  by  several  in¬ 
structors  with  remote  I/O  stations  in  the  classrooms, 
partially  by  research  teams  (sometimes  to  monitor 
physical  experiments  in  real  time),  partially  for  grad¬ 
uate  thesis  work,  and  partially  by  the  institution  for 
accounting,  payroll,  etc.  As  single  utility  systems  come 
to  provide  more  and  better  services  than  many  small 
machines,  perhaps  one  of  the  most  obvious  locations 
for  a  military  prototype  system  would  be  in  the  Pen¬ 
tagon.  A  Pentagon  prototype  utility  system  that  would 
serve  all  the  services  would  call  for  a  new  level  of 
cooperation  among  the  users  and  might  serve  as  a 
useful  guide  for  military  utility  systems  elsewhere.  Of 
course,  a  Pentagon  utility  system  may  be  seriously 
constrained  should  it  be  necessary  to  provide  for  se¬ 
cure  transfer  of  information  throughout  the  building. 
For  military  applications,  the  task  of  providing  high 
security  for  the  transfer  of  high  data  rate  digital  in¬ 
formation  is  one  demanding  early,  economical  solution. 
The  technology  to  handle  this  task  is  in  hand;  but,  as 
yet,  it  is  not  as  economical  as  it  must  become. 

In  at  least  one  case,  preliminary  results  concerning 
a  utility  system  indicate  that  it  is  efficient  for  the  user 
in  providing  low  cost  real-time  service  comparable  to 
that  of  an  on-site  installation  (C&A,  January  1966). 
Will  utility  systems  afford  the  military  user  the  flex¬ 
ibility  needed?  The  answer  to  this  question  is  closely 
intertwined  with  military  missions,  especially  during 
crises  and/or  wartime.  In  commercial  or  educational 
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utility  installations,  it  is  conceivable  that  a  crisis  affect¬ 
ing  one  or  several  users  of  the  system  would  have 
little  effect  on  many  of  the  other  users,  or  under  cer¬ 
tain  circumstances  various  users  could  reduce  their 
activities.  Almost  the  converse  is  true  for  the  military 
users.  A  crisis  that  affects  one  military  service  is  apt 
to  have  a  similar  effect  on  other  military  users  sharing 
the  system,  thus  bringing  demands  by  all  users  to  a 
peak.  Difficult  though  it  may  be  to  demonstrate  by 
means  of  cost-effectiveness,  the  military  need  for  flexi¬ 
bility  in  time  of  stress  may  surmount  what  appear  to  be 
the  obvious  advantages  to  be  gained  in  efficiency  of  op¬ 
eration  that  may  seem  to  be  afforded  by  use  of  utility 
systems.  Of  course,  the  utility  system  might  be  de¬ 
signed  to  accommodate  any  expected  peak  loads.  How¬ 
ever,  often  it  is  possible  to  generate  meaningful  system 
design  tests  only  by  actual  crisis  operation,  where  a 
design  failure  may  be  found  too  late  and  prove  to  be 
catastrophic. 

Another  potential  application  for  the  utility  system 
concept  is  at  the  base  level.  Today  almost  all  major 
military  bases  arc  apt  to  have  two  or  more  separate 
computer  installations.  As  more  and  more  functional 
military  areas — maintenance,  medicine,  communica¬ 
tions,  etc. — turn  to  data  automation  for  improved  op¬ 
erational  performance,  the  number  of  computers  at 
each  base  may  greatly  increase — unless  the  additional 
capability  can  be  provided  by  a  base  data  automation 
utility  system.  Of  course,  here  again  loom  the  twin 
specters  of  system  flexibility  and  survivability.  As  the 
nation’s  military  policy  continues  to  swing  toward 
developing  a  capability  for  mobility  and  responsiveness 
of  more  and  more  units  in  order  to  make  credible 
the  concept  of  controlled,  escalatory  warfare,  even 
should  that  mean  prolonged,  controlled  nuclear  con¬ 
flict,  then  it  may  develop  that  the  computers  used  for 
many  functions  at  base  level  will  be  required  to  be 
mobile  and  many  functions  normally  performed  at  the 
base  would  move  on  short  notice  to  remote  locations 
during  time  of  crisis  or  conflict. 

Thus  far,  the  comments  on  utility  systems  have  cen¬ 
tered  primarily  on  systems  that  for  the  most  part  might 
be  used  in  relatively  benign  environments,  e.g.,  in  the 
ZI.  As  indicated  earlier,  there  appears  to  be  a  place 
for  utility  systems  in  the  hands  ot  tactical  forces  in  the 
field  and  these  systems  would  likely  apply  to  the  com¬ 
plete  spectrum  of  possible  uses.  Aboard  ships,  warn¬ 
ing  and  control  aircraft,  and  command  post  aircraft  the 
local  environment  seems  to  favor  the  local  utility  con¬ 
cept  to  a  considerable  degree,  primarily  because  of  the 
ease  with  which  the  communications  and  security 
problems  can  be  handled.  For  ground  forces  in  the 
field,  communications,  security,  and  (perhaps  more 


important)  the  vulnerability  of  one  or  a  few  central 
data  processing  locations  tends  to  auger  against  the 
use  of  a  utility  system  for  even  moderately  large  geo¬ 
graphic  areas.  Within  Army  tactical  operation  centers 
and  like  points  of  management  and  control,  the  utility 
concept  will  likely  be  limited  to  a  large  central  data 
processor  and  I/O  stations  in  the  immediate  vicinity. 
In  general  military  operations  in  the  field — by  in¬ 
dividual  unit  and  often  by  function — are  apt  to  demand 
organic  data  processing  equipment  at  many  echelons 
to  achieve  the  flexibility,  security,  and  survivability 
needed. 

Development  of  future  systems 

The  statements  above  can  be  construed  to  imply  that 
much  of  the  important  structure  of  future  military  data 
automation  systems  is  fuzzy  at  best.  Also,  one  of  the 
most  important  questions  for  all  systems  would  seem  to 
be:  “Who  gets  the  central  processing  units?”  Some 
insight  to  this  part  of  planning  might  be  provided  by 
well-coordinated  tri-service  experimental  tests.  It  is  the 
writer’s  belief  that  the  test  should  be  carried  out  in  the 
field  (which  may  mean  aboard  ship,  at  a  specified 
headquarters,  a  military  base,  in  a  test  aircraft,  etc.) 
and  in  conjunction  with  the  potential  users  of  the  sys¬ 
tems.  In  some  cases  it  may  be  desirable  to  establish 
parallel  development  efforts,  one  at  a  field  installation 
and  one  at  a  research  or  development  center,  with  co¬ 
ordination  insured.  Coordination  is  also  required  among 
the  services  so  that  hardware  and  software  advances  by 
any  service  can  be  exploited  as  soon  as  possible  wherever 
they  arc  applicable. 

Listed  below  are  some  features  that  might  contribute 
to  a  workable,  productive  development  and  test  pro¬ 
gram.  Doubtless,  other  items  could  be  added. 

•  Top-level  support ,  direction,  and  guidance .  A 
tri-servicc  program  needs  support  from  a  cog¬ 
nizant  DOD  organization,  from  headquarters 
staff  level  in  each  service,  and  from  the  com¬ 
mander  of  the  military  organization  providing 
facilities  for  the  tests  performed  by  user  and 
development  personnel. 

•  A  user  group  to  support  the  test .  Test  success 
depends  greatly  on  experienced,  qualified  person¬ 
nel,  and  for  data  automation  in  military  tasks,  a 

user  organization  is  essential  to  test  the  utility  of 
the  system. 

•  Expert  help  from  contractors .  To  insure  optimal 
testing,  hardware  and  software  know-how  pro¬ 
vided  by  contractual  support  should  be  incor¬ 
porated,  but  with  control  in  the  hands  of  the  user, 
with  appropriate  guidance  and  assistance  from 
development  agencies. 
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•  Off-the-shelf  equipment.  Third-generation  data 
automation  equipment  should  be  used  when  pos¬ 
sible  to  provide  operational  experience  that  eould 
lead  to  a  better  basis  for  determining  more  mean¬ 
ingful  military  requirements.  It  also  would  eon- 
tribute  to  useful  feedback  and  cross-fertilization 
among  the  services,  the  development  agencies, 
and  the  planning  structure  in  the  DOD. 

•  Test  to  serve  at  least  one  need  thoroughly.  A  test 
need  not  attempt  to  serve  all  identifiable  needs, 
but  it  should  be  directed  toward  the  adequate 
solution  of  at  least  one  outstanding  task. 

•  Data  automation  experiment  as  an  aid ,  not  an  end. 
Each  test  and  each  development  effort  should 
reflect  the  faet  that  data  automation  should  be 
applied  to  aiding  man  in  his  military  duties,  rather 
than  attempting  to  replace  him. 

•  Means  for  the  exchange  of  experience  and  in¬ 
formation.  A  scries  of  on-site  field  tests  and  ex¬ 
periments  involving  user  personnel  can  be  most 
useful  if  all  parties  concerned  are  continuously 
kept  informed  of  all  parts  of  the  test  program. 
For  example,  an  airborne  data  processing  system 
applied  to  strategic  eommand  and  control  eould 
have  application  also  for  ground  and  shipboard 
command  and  control  systems  and,  more  broad¬ 
ly,  for  other  management  and  support  systems 
where  mobility  and  weight  are  of  prime  considera¬ 
tion.  Furthermore,  software  techniques  useful  in 
command  and  control  systems  may  find  at  least 
partial  application  in  medical  systems,  and  viee 

versa.  Channels  for  publishing  interim  results  and 
meetings  to  exchange  information  and  cxpericnee 
should  be  frequent. 

The  points  above  represent,  of  course,  but  a  be¬ 
ginning  in  the  outline  of  a  comprehensive  field  test  and 
development  program.  Central  to  the  entire  theme, 
obviously,  is  the  need  for  top-level  coordination,  wheth¬ 
er  the  program  is  undertaken  by  a  single  serviee  and 
its  own  agencies  and  commands,  or  whether  it  is 
handled  on  a  DOD-widc  basis.  Properly  coordinated 
at  either  level,  the  results  of  such  a  program  would  be 
of  inestimable  benefit  to  the  planners  of  future  systems. 

Potential  pitfalls  and  bottlenecks 

in  planning  for  future  systems 

While  many  of  the  points  to  be  made  below  have 
been  brought  out  elsewhere  in  this  paper,  a  few  of 
the  more  important  pitfalls  and  bottlenecks  in  master 
planning  are  explicitly  noted  here  for  convenience.  Fore¬ 
most  among  these  is  the  possibility  of  laek  of  upper 
echelon  guidance  and  coordination  of  master  plans  de¬ 
veloped  by  lower  levels  of  eommand.  Planners  at  lower 


levels  may  find  many  months  of  effort  turned  aside  by 
decisions  made  at  higher  levels  eoneerning,  for  ex¬ 
ample,  the  manner  in  whieh  utility  systems  may  be 
used  in  the  future.  As  another  example,  decisions  eon¬ 
eerning  the  use  of  data  processing  aboard  most  of  the 
Navy’s  First-line  ships  or  in  the  majority  of  the  Army’s 
ground  units  should  be  made  in  elosc  eonjunetion  be¬ 
tween  using  commands  and  higher  headquarters  and 
coordinated  throughout  the  DOD,  as  has  been  reiterated 
throughout  this  paper. 

Laek  of  field  experimentation  may  prove  to  be  a 
bottleneck  in  developing  meaningful  master  plans.  At 
the  moment  this  comment  applies  equally  to  the  ques¬ 
tion  of  the  general  military  applicability  of  the  utility 
system,  as  well  as  to  the  application  of  data  automation 
to  tactical  functions. 

In  spite  of  the  faet  that  communications  technical 
feasibility  is  not  in  question,  it  may  develop  that  certain 
communications  systems  will  for  a  time  be  inadequate 
for  the  many  data  automation  systems  that  may  be 
scheduled  for  their  use  in  master  plans.  Often  the  master 
planners  at  lower  echelons  look  upon  a  communication 
system  such  as  might  be  found  at  base  level  or  at  higher 
level,  such  as  AUTODIN,  as  the  expeeted  means  for 
transmission  of  information.  As  was  the  ease  with  the 
early  users  of  AUTODIN,  the  data  automation  master 
planner  may  find,  to  his  surprise  and  shock,  that  not 
only  is  much  of  the  communication  system’s  capacity 
being  used  by  others  but  also  that  the  system  itself 
exhibits  certain  characteristics  that  seriously  curtail  ef¬ 
fective  data  rate. 

It  may  develop  in  newly  applied  data  automation  sys¬ 
tems  that  the  operational  unit  is  inadequately  organized 
and/or  staffed  to  bring  the  system  into  full  operational 
capability  in  the  expeeted  period  of  time.  A  pitfall  of 
this  kind  ean  be  expeeted  as  data  automation  is  more 
widely  applied  to  new  functional  areas  such  as  those 
eneompassed  by  the  surgeon  general,  the  inspector  gen¬ 
eral,  the  judge  advoeate,  ete.  And  it  is  likely  to  be  even 
more  prevalent  in  the  application  of  data  automation 
in  the  taetical  area.  Limited-scope  field  tests  and  ex¬ 
periments  might  tend  to  ease  this  potential  bottleneck. 

Two  well-known  workhorses  conclude  this  list  of 
pitfalls:  One  is  standardization  of  data  processing  lan¬ 
guages,  and  little  more  will  be  said  except  that  it  is 
needed,  for  it  is  getting  attention  today  and  it  will  doubt¬ 
less  require  additional  attention  throughout  the  fore¬ 
seeable  future.  The  seeond  well-known  bottleneck  is 
training:  Data  automation  training  continues  to  be 
needed  in  all  functional  areas  of  application  and  at  all 
levels  of  eommand.  All  master  plans  should  inelude 
an  adequate  treatment  of  an  aeeompanying  training 
program.  Fortunately,  advances  in  computer  software 
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are  leading  to  programming  languages  that  are  increas¬ 
ingly  easier  to  master.  And  there  is  always  the  hope 
that  new  data  automation  systems  will  come  equipped 
with  a  programmed  learning  feature,  so  that  the  opera¬ 
tion,  programming,  and  maintenance  of  the  equipment 
can  be  learned  in  programmed  fashion  from  the  ma¬ 
chine  itself.  A  feature  such  as  this  would  be  of  great 
benefit,  if  it  could  be  made  a  part  of  tactical  data  auto¬ 
mation  systems,  where  the  turnover  of  personnel  in  the 
unit  may  be  high. 

Suggestions  for  future  planning 

The  central  themes  of  this  paper  are  few  and  simple. 
In  summary  form,  reworded  as  suggestions  for  future 
planning  efforts,  they  might  be  as  follows: 

•  In  all  master  planning  efforts,  insist  on  receiving 
guidance  and  coordination  from  above  and  in¬ 
sure  that  it  is  given  to  echelons  below. 

•  Where  experience  for  master  planning  is  lacking, 
outline  on-site,  user-performed  tests  and  experi¬ 
ments  to  provide  the  experience  and  insight  re¬ 
quired.  In  field  tests,  make  use  of  existing  hard¬ 
ware  (and  software,  where  possible)  and  con¬ 
tractor  support,  but  keep  the  user  in  control. 


•  In  planning  for  military  data  automation  sys¬ 
tems,  do  not  lose  sight  of  the  need  to  keep  mili¬ 
tary  systems  flexible  and  survivable;  in  the  future 
this  may  impose  requirements  of  redundancy 
and  mobility  on  systems  that  today  are  con¬ 
sidered  to  be  of  the  fixed  installation  type. 

•  Keep  in  clear  view  the  pitfalls  and  bottlenecks 
that  may  plague  the  data  automation  systems 
proposed  in  master  plans  and  recognize  that 
some  of  the  difficulties  may  be  alleviated  by 
adequate  organization  and  staffing  of  the  opera¬ 
tional  units,  by  adequate  training  at  all  levels  of 
command,  and  by  communication  systems  that 
are  compatible  with  the  data  processing  system 
and  adequate  to  serve  all  demands. 
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